Intrinsic motivation for choice varies with individual risk attitudes and the controllability of the environment.

When deciding between options that do or do not lead to future choices, humans often choose to choose. We studied choice seeking by asking subjects to first decide between a choice opportunity or performing a computer-selected action, after which they either chose freely or performed the forced action. Subjects preferred choice when these options were equally rewarded, even deterministically, and traded extrinsic rewards for opportunities to choose. We explained individual variability in choice seeking using reinforcement learning models incorporating risk sensitivity and overvaluation of rewards obtained through choice. Model fits revealed that 28% of subjects were sensitive to the worst possible outcome associated with free choice, and this pessimism reduced their choice preference with increasing risk. Moreover, outcome overvaluation was necessary to explain patterns of individual choice preference across levels of risk. We also manipulated the degree to which subjects controlled stimulus outcomes. We found that degrading coherence between their actions and stimulus outcomes diminished choice preference following forced actions, although willingness to repeat selection of choice opportunities remained high. When subjects chose freely during these repeats, they were sensitive to rewards when actions were controllable but ignored outcomes-even positive ones-associated with reduced controllability. Our results show that preference for choice can be modulated by extrinsic reward properties including reward probability and risk as well as by controllability of the environment.

Correction: Intrinsic motivation for choice varies with individual risk attitudes and the controllability of the environment.

[This corrects the article DOI: 10.1371/journal.pcbi.1010551.].

Motor Coordination and Strategic Cooperation in Joint Action.

Naturalistic joint action between two agents typically requires both motor coordination and strategic cooperation. However, these two fundamental processes have systematically been studied independently. We presented 50 dyads of adult participants with a novel collaborative task that combined different levels of motor noise with different levels of strategic noise, to determine whether the sense of agency (the experience of control over an action) reflects the interplay between these low-level (motor) and high-level (strategic) dimensions. We also examined how dominance in motor control could influence prosocial behaviors. We found that self-agency was particularly dependent on motor cues, whereas joint agency was particularly dependent on strategic cues. We suggest that the prime importance of strategic cues to joint agency reflects the co-representation of coagents' interests during the task. Furthermore, we observed a reduction in prosocial strategies in agents who exerted dominant motor control over joint action, showing that the strategic dimension of human interactions is also susceptible to the influence of low-level motor characteristics.

Focal versus distributed temporal cortex activity for speech sound category assignment.

Percepts and words can be decoded from distributed neural activity measures. However, the existence of widespread representations might conflict with the more classical notions of hierarchical processing and efficient coding, which are especially relevant in speech processing. Using fMRI and magnetoencephalography during syllable identification, we show that sensory and decisional activity colocalize to a restricted part of the posterior superior temporal gyrus (pSTG). Next, using intracortical recordings, we demonstrate that early and focal neural activity in this region distinguishes correct from incorrect decisions and can be machine-decoded to classify syllables. Crucially, significant machine decoding was possible from neuronal activity sampled across different regions of the temporal and frontal lobes, despite weak or absent sensory or decision-related responses. These findings show that speech-sound categorization relies on an efficient readout of focal pSTG neural activity, while more distributed activity patterns, although classifiable by machine learning, instead reflect collateral processes of sensory perception and decision.

Cost-benefit trade-offs in decision-making and learning.

Value-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Moreover, engaging control to proactively suppress irrelevant information that could conflict with task-relevant information would presumably also be cognitively costly. Yet, it remains unclear whether the cognitive control demands involved in preventing and resolving conflict also constitute costs in value-based decisions. The present study investigated this question by embedding irrelevant distractors (flanker arrows) within a reversal-learning task, with intermixed free and instructed trials. Results showed that participants learned to adapt their free choices to maximize rewards, but were nevertheless biased to follow the suggestions of irrelevant distractors. Thus, the perceived cost of investing cognitive control to suppress an external suggestion could sometimes trump internal value representations. By adapting computational models of reinforcement learning, we assessed the influence of conflict at both the decision and learning stages. Modelling the decision showed that free choices were more biased when participants were less sure about which action was more rewarding. This supports the hypothesis that the costs linked to conflict management were traded off against expected rewards. During the learning phase, we found that learning rates were reduced in instructed, relative to free, choices. Learning rates were further reduced by conflict between an instruction and subjective action values, whereas learning was not robustly influenced by conflict between one's actions and external distractors. Our results show that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and highlight that different types of conflict may have different effects on learning about action outcomes.

In and out of control: brain mechanisms linking fluency of action selection to self-agency in patients with schizophrenia.

Sense of agency refers to the feeling of control over one's actions, and their consequences. It involves both predictive processes linked to action control, and retrospective 'sense-making' causal inferences. Schizophrenia has been associated with impaired predictive processing, but the underlying mechanisms that impair patients' sense of agency remain unclear. We introduce a new 'prospective' aspect of agency and show that subliminally priming an action not only influences response times, but also influences reported sense of agency over subsequent action outcomes. This effect of priming was associated with altered connectivity between frontal areas and the angular gyrus. The effects on response times and on frontal action selection mechanisms were similar in patients with schizophrenia and in healthy volunteers. However, patients showed no effects of priming on sense of agency, no priming-related activation of angular gyrus, and no priming-related changes in fronto-parietal connectivity. We suggest angular gyrus activation reflects the experiences of agency, or non-agency, in part by processing action selection signals generated in the frontal lobes. The altered action awareness that characterizes schizophrenia may be due to impaired communication between these areas.

An online neural substrate for sense of agency.

"Sense of agency" refers to the feeling of controlling an external event through one's own action. On one influential view, sense of agency is inferred after an action, by "retrospectively" comparing actual effects of actions against their intended effects. In contrast, a "prospective" component of agency, generated during action selection, and in advance of knowing the actual effect, has received less attention. Here, we used functional magnetic resonance imaging to investigate prospective contributions of action selection processes to sense of agency. To do so, we dissociated action selection processes from action-outcome matching, by subliminally priming responses to a target. We found that participants experienced greater control over action effects when the action was compatibly versus incompatibly primed. Thus, compatible primes facilitated action selection processing, in turn boosting sense of agency over a subsequent effect. This prospective contribution of action selection processes to sense of agency was accounted for by exchange of signals across a prefrontal-parietal network. Specifically, we found that the angular gyrus (AG) monitors signals relating to action selection in dorsolateral prefrontal cortex, to prospectively inform subjective judgments of control over action outcomes. Online monitoring of these signals by AG might provide the subject with a subjective marker of volition, prior to action itself.

Priming of actions increases sense of control over unexpected outcomes.

Sense of agency (SoA) refers to the feeling that we are in control of our own actions and, through them, events in the outside world. SoA depends partly on retrospectively matching outcomes to expectations, and partly on prospective processes occurring prior to action, notably action selection. To assess the relative contribution of these processes, we factorially varied subliminal priming of action selection and expectation of action outcomes. Both factors affected SoA, and there was also a significant interaction. Compatible action primes increased SoA more strongly for unexpected than expected outcomes. Outcome expectation had strong effects on SoA following incompatible action priming, but only weak effects following compatible action priming. Prospective and retrospective SoA may have distinct and complementary functions.

The reliability of assistance systems modulates the sense of control and acceptability of human operators.

Individuals are increasingly required to interact with complex and autonomous technologies, which often has a significant impact on the control they experience over their actions and choices. A better characterization of the factors responsible for modulating the control experience of human operators is therefore a major challenge to improve the quality of human-system interactions. Using a decision-making task performed in interaction with an automated system, we investigated the influence of two key properties of automated systems, their reliability and explicability, on participants' sense of agency (SoA), as well as the perceived acceptability of system choices. The results show an increase in SoA associated with the most explicable system. Importantly, the increase in system explicability influenced participants' ability to regulate the control resources they engaged in the current decision. In particular, we observed that participants' SoA varied with system reliability in the "explained" condition, whereas no variation was observed in the "non-explained" condition. Finally, we found that system reliability had a direct impact on system acceptability, such that the most reliable systems were also considered the most acceptable systems. These results highlight the importance of studying agency in human-computer interaction in order to define more acceptable automation technologies.

Associations Between Early Life Adversity, Reproduction-Oriented Life Strategy, and Borderline Personality Disorder.

Importance: Borderline personality disorder (BPD) is often accompanied by a history of high-risk sexual behavior and somatic comorbidities. Yet, these features are most often considered in isolation and little is known about their underlying developmental pathways. Life history theory, a leading framework in evolutionary developmental biology, can help make sense of the wide range of behaviors and health issues found in BPD. Objective: To examine whether the emergence of BPD is associated with the prioritization of immediate reproductive goals over longer-term somatic maintenance goals, a life strategy that can be viewed as a developmental response to adverse early life experiences, providing rapid reproductive benefits despite costs to health and well-being. Design, Setting, and Participants: This study used cross-sectional data from the second wave of the National Epidemiologic Survey on Alcohol and Related Conditions in 2004-2005 (n = 34 653). Civilian, noninstitutionalized individuals in the US, 18 years or older, and those with and without a DSM-IV diagnosis of BPD were included. Analysis took place between August 2020 and June 2021. Main Outcomes and Measures: Structural equation models were used to examine whether early life adversity was associated with the likelihood of a BPD diagnosis, either directly or indirectly through a life strategy whereby individuals trade somatic maintenance for immediate reproduction. Results: Analyses were performed on a sample of 30 149 participants (females: 17 042 [52%]; mean [SE] age, 48.5 [0.09]; males: 12 747 [48%]; mean [SE] age, 47 [0.08]). Of these, 892 (2.7%) had a diagnosis of BPD and 29 257 (97.3%) did not have BPD. Mean early life adversity, metabolic disorder score, and body mass index were significantly higher among participants with a diagnosis of BPD. In an analysis adjusted for age, individuals with BPD reported having significantly more children than those without BPD (b =0.06; SE, 0.01; t = 4.09; P < .001). Having experienced greater levels of adversity in early life was significantly associated with a greater risk of being diagnosed with BPD later in life (direct relative risk = 0.268; SE, 0.067; P < .001). Importantly, this risk was further increased by 56.5% among respondents who prioritized short-term reproductive goals over somatic maintenance (indirect relative risk = 0.565; SE, 0.056; P < .001). Similar patterns of associations were found in male and female individuals. Conclusions and Relevance: The hypothesis of a reproduction/maintenance life history trade-off mediating the association between early life adversity and BPD helps make sense of the high dimensionality that characterizes the physiological and behavioral correlates of BPD. Additional studies are needed to confirm these results using longitudinal data.

Mentalizing under influence: abnormal dependence on prior expectations in patients with schizophrenia.

An impaired ability to appreciate other people's mental states is a well-established and stable cognitive deficit in schizophrenia, which might explain some aspects of patients' social dysfunction. Yet, despite a wealth of literature on this topic, the basic mechanisms underlying these impairments are still poorly understood, and their links with the clinical dimensions of schizophrenia remain unclear. The present study aimed to investigate the extent to which patients' impaired ability to appreciate other people's intentions (known as mentalizing) may be accounted for by abnormal interaction between the two types of information that contribute to this ability: (i) the sensory evidence conveyed by movement kinematics; and (ii) the observer's prior expectations. We hypothesized that this is not a generalized impairment, but one confined to certain types of intentions. To test this assumption, we designed four tasks in which participants were required to infer either: (i) basic intentions (i.e. the simple goal of a motor act); (ii) superordinate intentions (i.e. the general goal of a sequence of motor acts); (iii) social basic; or (iv) social superordinate intentions (i.e. simple or general goals achieved within the context of a reciprocal interaction). In each of these tasks, both prior expectations and sensory information were manipulated. We found that patients correctly inferred non-social, basic intentions, but experienced difficulties when inferring non-social superordinate intentions and both basic and superordinate social intentions. These poor performances were associated with two abnormal patterns of interaction between prior expectations and sensory evidence. In the non-social superordinate condition, patients relied heavily on their prior expectations, while disregarding sensory evidence. This pattern of interaction predicted the severity of 'positive' symptoms. Social conditions prompted exactly the opposite pattern of interaction: patients exhibited weaker dependence on prior expectations while relying strongly on sensory evidence, and this predicted the severity of 'negative' symptoms. We suggest both these patterns can be accounted for by a disturbance in the Bayesian inferential mechanism that integrates sensory evidence (conveyed by movement kinematics) into prior beliefs (about others' mental states and attitudes) to produce accurate inferences about other people's intentions.

Leveraging human agency to improve confidence and acceptability in human-machine interactions.

Repeated interactions with automated systems are known to affect how agents experience their own actions and choices. The present study explores the possibility of partially restoring sense of agency in operators interacting with automated systems by providing additional information about the system's decision, i.e. its confidence. To do so, we implemented an obstacle avoidance task with different levels of automation and explicability. Levels of automation were varied by implementing conditions in which the participant was free or not free to choose which direction to take, whereas levels of explicability were varied by providing or not providing the participant with the system's confidence in the direction to take. We first assessed how automation and explicability interacted with participants' sense of agency, and then tested whether increased self-agency was systematically associated with greater confidence in the decision and improved system acceptability. The results showed an overall positive effect of system assistance. Providing additional information about the system's decision (explicability effect) and reducing the cognitive load associated with the decision itself (automation effect) was associated with stronger sense of agency, greater confidence in the decision, and better performance. In addition to the positive effects of system assistance, acceptability scores revealed that participants perceived "explicable" systems more favorably. These results highlight the potential value of studying self-agency in human-machine interaction as a guideline for making automation technologies more acceptable and, ultimately, improving the usefulness of these technologies.

What is new with Artificial Intelligence? Human-agent interactions through the lens of social agency.

In this article, we suggest that the study of social interactions and the development of a "sense of agency" in joint action can help determine the content of relevant explanations to be implemented in artificial systems to make them "explainable." The introduction of automated systems, and more broadly of Artificial Intelligence (AI), into many domains has profoundly changed the nature of human activity, as well as the subjective experience that agents have of their own actions and their consequences - an experience that is commonly referred to as sense of agency. We propose to examine the empirical evidence supporting this impact of automation on individuals' sense of agency, and hence on measures as diverse as operator performance, system explicability and acceptability. Because of some of its key characteristics, AI occupies a special status in the artificial systems landscape. We suggest that this status prompts us to reconsider human-AI interactions in the light of human-human relations. We approach the study of joint actions in human social interactions to deduce what key features are necessary for the development of a reliable sense of agency in a social context and suggest that such framework can help define what constitutes a good explanation. Finally, we propose possible directions to improve human-AI interactions and, in particular, to restore the sense of agency of human operators, improve their confidence in the decisions made by artificial agents, and increase the acceptability of such agents.

Susceptibility of agency judgments to social influence.

The experience of agency refers to the phenomenal experience of being the causal source of one's own actions, and through them, the course of events in the outside world. This experience is crucial for the production of adaptive actions, and for the adequate communication of felt action control to peers. The present study examines the possibility that, on certain occasions and under specific internal and external constraints, people rely on explicit social information provided by their peers to revise their self-reports of perceived control, i.e., their judgment of agency. To test this hypothesis, we adapted a task based on an interactive computer game. We manipulated well-known sensorimotor agency cues related to action control, as well as social information communicated to participants by two advisors. We measured the contribution of social and non-social sources of information to agency judgments. We found that at the single-trial level, participants align their JoA with advisor feedback based on their own performance during the task, the type of feedback provided by advisors, and the interaction of this social feedback with the sensorimotor agency cues. At the same time, JoA alignment in previous trial also predicted participants' tendency to revise their JoA after social feedback. Overall, these results demonstrate that agency judgment is subject to social influence. This influence is the result of the integration of social and non-social information at the scale of a single judgment, while also being driven by repeated past interactions with peers.

Controllability boosts neural and cognitive signatures of changes-of-mind in uncertain environments.

In uncertain environments, seeking information about alternative choice options is essential for adaptive learning and decision-making. However, information seeking is usually confounded with changes-of-mind about the reliability of the preferred option. Here, we exploited the fact that information seeking requires control over which option to sample to isolate its behavioral and neurophysiological signatures. We found that changes-of-mind occurring with control require more evidence against the current option, are associated with reduced confidence, but are nevertheless more likely to be confirmed on the next decision. Multimodal neurophysiological recordings showed that these changes-of-mind are preceded by stronger activation of the dorsal attention network in magnetoencephalography, and followed by increased pupil-linked arousal during the presentation of decision outcomes. Together, these findings indicate that information seeking increases the saliency of evidence perceived as the direct consequence of one's own actions.

Interacting with volatile environments stabilizes hidden-state inference and its brain signatures.

Making accurate decisions in uncertain environments requires identifying the generative cause of sensory cues, but also the expected outcomes of possible actions. Although both cognitive processes can be formalized as Bayesian inference, they are commonly studied using different experimental frameworks, making their formal comparison difficult. Here, by framing a reversal learning task either as cue-based or outcome-based inference, we found that humans perceive the same volatile environment as more stable when inferring its hidden state by interaction with uncertain outcomes than by observation of equally uncertain cues. Multivariate patterns of magnetoencephalographic (MEG) activity reflected this behavioral difference in the neural interaction between inferred beliefs and incoming evidence, an effect originating from associative regions in the temporal lobe. Together, these findings indicate that the degree of control over the sampling of volatile environments shapes human learning and decision-making under uncertainty.

The Cognitive, Ecological, and Developmental Origins of Self-Disturbance in Borderline Personality Disorder.

Self-disturbance is recognized as a key symptom of Borderline Personality Disorder (BPD). Although it is the source of significant distress and significant costs to society, it is still poorly specified. In addition, current research and models on the etiology of BPD do not provide sufficient evidence or predictions about who is at risk of developing BPD and self-disturbance, and why. The aim of this review is to lay the foundations of a new model inspired by recent developments at the intersection of social cognition, behavioral ecology, and developmental biology. We argue that the sense of agency is an important dimension to consider when characterizing self-disturbances in BPD. Second, we address the poorly characterized relation between self-disturbances and adverse life conditions encountered early in life. We highlight the potential relevance of Life-History Theory-a major framework in evolutionary developmental biology-to make sense of this association. We put forward the idea that the effect of early life adversity on BPD symptomatology depends on the way individuals trade their limited resources between competing biological functions during development.

Agents' pivotality and reward fairness modulate sense of agency in cooperative joint action.

The sense of agency (SoA) experienced in joint action is an essential subjective dimension of human cooperativeness, but we still know little about the specific factors that contribute to its emergence or alteration. In the present study, dyads of participants were instructed to coordinate their key presses to move a cursor up to a specific target (i.e., to achieve a common goal). We applied random deviations on the cursor's trajectory to manipulate the motor fluency of the joint action, while the agents' motor roles were either balanced (i.e., equivalent) or unbalanced (i.e., one agent contributed more than the other), making the agents more or less pivotal to the joint action. Then, the final outcomes were shared equally, fairly (i.e., reflecting individual motor contributions) or arbitrarily in an all-or-none fashion, between the co-agents. Self and joint SoA were measured through self-reports about feeling of control, that is, using judgment of (felt) control (JoC), and electrodermal activity was recorded during the whole motor task. We observed that self and joint JoC were reduced in the case of low motor fluency, pointing out the importance of sensorimotor cues for both I- and we-modes. Moreover, while self JoC was reduced in the low pivotality condition (i.e., low motor role), joint JoC was significantly enhanced when agents' roles and rewards were symmetrical (i.e. equal). Skin conductance responses to rewards were impacted by the way outcomes were shared between partners (i.e., fairly, equally or arbitrarily) but not by the individual gains, which demonstrates the sensitivity of low-level physiological reactions to external signs of fairness. Skin conductance level was also reduced in the fair context, where rewards were shared according to individual motor contributions, relative to the all-or-none context, which could mirror the feeling of effective responsibility and control over actions' outcomes.

Information about action outcomes differentially affects learning from self-determined versus imposed choices.

The valence of new information influences learning rates in humans: good news tends to receive more weight than bad news. We investigated this learning bias in four experiments, by systematically manipulating the source of required action (free versus forced choices), outcome contingencies (low versus high reward) and motor requirements (go versus no-go choices). Analysis of model-estimated learning rates showed that the confirmation bias in learning rates was specific to free choices, but was independent of outcome contingencies. The bias was also unaffected by the motor requirements, thus suggesting that it operates in the representational space of decisions, rather than motoric actions. Finally, model simulations revealed that learning rates estimated from the choice-confirmation model had the effect of maximizing performance across low- and high-reward environments. We therefore suggest that choice-confirmation bias may be adaptive for efficient learning of action-outcome contingencies, above and beyond fostering person-level dispositions such as self-esteem.

The architecture of cognitive control in schizophrenia.

Executive dysfunctions have long been considered a common feature of schizophrenia. However, due to their extreme heterogeneity, it is not clear whether these impairments take place at a particular level of executive functioning or non-specifically affect various aspects of behavioural control. To answer this question, we used an experimental paradigm based upon a multistage model of prefrontal executive function. This model postulates that cognitive control is organized in three hierarchically ordered control processes, operating with respect to the perceptual context (sensory and contextual controls) or the temporal episode in which the person is acting (episodic control). Twenty-four patients with schizophrenia and 24 non-psychiatric controls participated in two distinct experiments designed to separately assess each of these three levels of control. The results indicate that both sensory and episodic dimensions of cognitive control were spared in schizophrenic patients, but that they showed great difficulty in contextual conditions, as the selection of the appropriate response among competitive ones required taking into account information related to perceptual context. Contextual control can be considered as a set of executive processes mediating the hierarchical organization of behaviour. Patients' deficit in cognitive control therefore reflects a specific problem in the hierarchical control of action, leading to the selection of inappropriate behavioural representations for ongoing action plans. We also showed that this impairment was a good predictor of disorganization syndrome scores, suggesting that these clinical manifestations might result from a deficit in the combination or selection of hierarchically organized action representations.