Cost-benefit trade-offs in decision-making and learning.

Value-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Moreover, engaging control to proactively suppress irrelevant information that could conflict with task-relevant information would presumably also be cognitively costly. Yet, it remains unclear whether the cognitive control demands involved in preventing and resolving conflict also constitute costs in value-based decisions. The present study investigated this question by embedding irrelevant distractors (flanker arrows) within a reversal-learning task, with intermixed free and instructed trials. Results showed that participants learned to adapt their free choices to maximize rewards, but were nevertheless biased to follow the suggestions of irrelevant distractors. Thus, the perceived cost of investing cognitive control to suppress an external suggestion could sometimes trump internal value representations. By adapting computational models of reinforcement learning, we assessed the influence of conflict at both the decision and learning stages. Modelling the decision showed that free choices were more biased when participants were less sure about which action was more rewarding. This supports the hypothesis that the costs linked to conflict management were traded off against expected rewards. During the learning phase, we found that learning rates were reduced in instructed, relative to free, choices. Learning rates were further reduced by conflict between an instruction and subjective action values, whereas learning was not robustly influenced by conflict between one's actions and external distractors. Our results show that the subjective cognitive control costs linked to conflict factor into value-based decision-making, and highlight that different types of conflict may have different effects on learning about action outcomes.

How action selection influences the sense of agency: An ERP study.

Sense of agency (SoA) refers to the feeling that we are in control of our actions and, through them, of events in the outside world. One influential view claims that the SoA depends on retrospectively matching the expected and actual outcomes of action. However, recent studies have revealed an additional, prospective component to SoA, driven by action selection processes. We used event-related potentials (ERPs) to clarify the neural mechanisms underlying prospective agency. Subliminal priming was used to manipulate the fluency of selecting a left or right hand action in response to a supraliminal target. These actions were followed by one of several coloured circles, after a variable delay. Participants then rated their degree of control over this visual outcome. Incompatible priming impaired action selection, and reduced sense of agency over action outcomes, relative to compatible priming. More negative ERPs immediately after the action, linked to post-decisional action monitoring, were associated with reduced agency ratings over action outcomes. Additionally, feedback-related negativity evoked by the outcome was also associated with reduced agency ratings. These ERP components may reflect brain processes underlying prospective and retrospective components of sense of agency respectively.

Influences of unconscious priming on voluntary actions: Role of the rostral cingulate zone.

The ability to make voluntary, free choices is fundamental to what it means to be human. A key brain region that is involved in free choices is the rostral cingulate zone (RCZ), which is part of the medial frontal cortex. Previous research has shown that activity in this brain region can be modulated by bottom-up information while making free choices. The current study extends those findings, and shows, for the first time, that activation in the RCZ can also be modulated by subliminal information. We used a subliminal response priming paradigm to bias free and cued choices. We observed more activation in the RCZ when participants made a choice that went against the prime's suggestion, compared to when they chose according to the prime. This shows that the RCZ plays an important role in overcoming externally-triggered conflict between different response options, even when the stimuli triggering this conflict are not consciously perceived. Our results suggest that an important mechanism of endogenous action in the RCZ may therefore involve exerting an internally-generated action choice against conflicting influences, such as external sensory evidence. We further found that subliminal information also modulated activity in the anterior insula and the supramarginal gyrus.

Priming of actions increases sense of control over unexpected outcomes.

Sense of agency (SoA) refers to the feeling that we are in control of our own actions and, through them, events in the outside world. SoA depends partly on retrospectively matching outcomes to expectations, and partly on prospective processes occurring prior to action, notably action selection. To assess the relative contribution of these processes, we factorially varied subliminal priming of action selection and expectation of action outcomes. Both factors affected SoA, and there was also a significant interaction. Compatible action primes increased SoA more strongly for unexpected than expected outcomes. Outcome expectation had strong effects on SoA following incompatible action priming, but only weak effects following compatible action priming. Prospective and retrospective SoA may have distinct and complementary functions.

Susceptibility of agency judgments to social influence.

The experience of agency refers to the phenomenal experience of being the causal source of one's own actions, and through them, the course of events in the outside world. This experience is crucial for the production of adaptive actions, and for the adequate communication of felt action control to peers. The present study examines the possibility that, on certain occasions and under specific internal and external constraints, people rely on explicit social information provided by their peers to revise their self-reports of perceived control, i.e., their judgment of agency. To test this hypothesis, we adapted a task based on an interactive computer game. We manipulated well-known sensorimotor agency cues related to action control, as well as social information communicated to participants by two advisors. We measured the contribution of social and non-social sources of information to agency judgments. We found that at the single-trial level, participants align their JoA with advisor feedback based on their own performance during the task, the type of feedback provided by advisors, and the interaction of this social feedback with the sensorimotor agency cues. At the same time, JoA alignment in previous trial also predicted participants' tendency to revise their JoA after social feedback. Overall, these results demonstrate that agency judgment is subject to social influence. This influence is the result of the integration of social and non-social information at the scale of a single judgment, while also being driven by repeated past interactions with peers.

Losing Control in Social Situations: How the Presence of Others Affects Neural Processes Related to Sense of Agency.

Social contexts substantially influence individual behavior, but little is known about how they affect cognitive processes related to voluntary action. Previously, it has been shown that social context reduces participants' sense of agency over the outcomes of their actions and outcome monitoring. In this fMRI study on human volunteers, we investigated the neural mechanisms by which social context alters sense of agency. Participants made costly actions to stop inflating a balloon before it burst. On "social" trials, another player could act in their stead, but we analyzed only trials in which the other player remained passive. We hypothesized that mentalizing processes during social trials would affect decision-making fluency and lead to a decreased sense of agency. In line with this hypothesis, we found increased activity in the bilateral temporo-parietal junction (TPJ), precuneus, and middle frontal gyrus during social trials compared with nonsocial trials. Activity in the precuneus was, in turn, negatively related to sense of agency at a single-trial level. We further found a double dissociation between TPJ and angular gyrus (AG): activity in the left AG was not sensitive to social context but was negatively related to sense of agency. In contrast, activity in the TPJ was modulated by social context but was not sensitive to sense of agency.

Integrating prospective and retrospective cues to the sense of agency: a multi-study investigation.

Sense of agency (SoA) refers to the experience of voluntary control over one's own actions, and, through them, over events in the outside world. Recent accounts suggest that SoA involves an integration of various cues. These include prospective cues, for example, related to the fluency of action selection, and retrospective cues, linked to outcome monitoring. It remains unclear whether these cues may have independent effects on SoA, and, in particular, how their relative contributions may change during instrumental learning. In the present study, we explored these issues by conducting a multi-study analysis of seven published and unpublished studies on the role of prospective cues to the SoA. Our main question was how the effects of selection fluency on SoA might change as information about action-outcome contingencies is gathered. Results show that selection fluency can have a general and consistent influence on the SoA, independent of outcome monitoring. This suggests selection fluency is used as a heuristic cue, to prospectively inform our SoA. In addition, our results show that the influence of selection fluency on SoA may change systematically as action-outcome contingencies are gradually learned. We speculate that dysfluent selection may impair formation of mental associations between action and outcome.

Investigating the Prospective Sense of Agency: Effects of Processing Fluency, Stimulus Ambiguity, and Response Conflict.

How do we know how much control we have over our environment? The sense of agency refers to the feeling that we are in control of our actions, and that, through them, we can control our external environment. Thus, agency clearly involves matching intentions, actions, and outcomes. The present studies investigated the possibility that processes of action selection, i.e., choosing what action to make, contribute to the sense of agency. Since selection of action necessarily precedes execution of action, such effects must be prospective. In contrast, most literature on sense of agency has focussed on the retrospective computation whether an outcome fits the action performed or intended. This hypothesis was tested in an ecologically rich, dynamic task based on a computer game. Across three experiments, we manipulated three different aspects of action selection processing: visual processing fluency, categorization ambiguity, and response conflict. Additionally, we measured the relative contributions of prospective, action selection-based cues, and retrospective, outcome-based cues to the sense of agency. Manipulations of action selection were orthogonally combined with discrepancy of visual feedback of action. Fluency of action selection had a small but reliable effect on the sense of agency. Additionally, as expected, sense of agency was strongly reduced when visual feedback was discrepant with the action performed. The effects of discrepant feedback were larger than the effects of action selection fluency, and sometimes suppressed them. The sense of agency is highly sensitive to disruptions of action-outcome relations. However, when motor control is successful, and action-outcome relations are as predicted, fluency or dysfluency of action selection provides an important prospective cue to the sense of agency.

Beyond self-serving bias: diffusion of responsibility reduces sense of agency and outcome monitoring.

Diffusion of responsibility across agents has been proposed to underlie decreased helping and increased aggression in group behaviour. However, few studies have directly investigated effects of the presence of other people on how we experience the consequences of our actions. This EEG study investigated whether diffusion of responsibility simply reflects a post-hoc self-serving bias, or rather has direct effects on how we process the outcomes of our actions, and our experience of agency over them. Participants made voluntary actions whose outcomes were more or less negative. Presence of another potential agent reduced participants' sense of agency over those outcomes, even though it was always obvious who caused each outcome. Further, presence of another agent reduced the amplitude of feedback-related negativity evoked by outcome stimuli, suggesting reduced outcome monitoring. The presence of other agents may lead to diffusion of responsibility by weakening the neural linkage between one's actions and their outcomes.

Difficult action decisions reduce the sense of agency: A study using the Eriksen flanker task.

The sense of agency refers to the feeling that we are in control of our actions and, through them, of events in the outside world. Much research has focused on the importance of retrospectively matching predicted and actual action outcomes for a strong sense of agency. Yet, recent studies have revealed that a metacognitive signal about the fluency of action selection can prospectively inform our sense of agency. Fluent, or easy, action selection leads to a stronger sense of agency over action outcomes than dysfluent, or difficult, selection. Since these studies used subliminal priming to manipulate action selection, it remained unclear whether supraliminal stimuli affecting action selection would have similar effects. We used supraliminal flankers to manipulate action selection in response to a central target. Experiment 1 revealed that conflict in action selection, induced by incongruent flankers and targets, led to reduced agency ratings over an outcome that followed the participant's response, relative to neutral and congruent flanking conditions. Experiment 2 replicated this result, and extended it to free choice between alternative actions. Finally, Experiment 3 varied the stimulus onset asynchrony (SOA) between flankers and target. Action selection performance varied with SOA. Agency ratings were always lower in incongruent than congruent trials, and this effect did not vary across SOAs. Sense of agency is influenced by a signal that tracks conflict in action selection, regardless of the visibility of stimuli inducing conflict, and even when the timing of the stimuli means that the conflict may not affect performance.

From action intentions to action effects: how does the sense of agency come about?

Sense of agency refers to the feeling of controlling an external event through one's own action. On one influential view, agency depends on how predictable the consequences of one's action are, getting stronger as the match between predicted and actual effect of an action gets closer. Thus, sense of agency arises when external events that follow our action are consistent with predictions of action effects made by the motor system while we perform or simply intend to perform an action. According to this view, agency is inferred retrospectively, after an action has been performed and its consequences are known. In contrast, little is known about whether and how internal processes involved in the selection of actions may influence subjective sense of control, in advance of the action itself, and irrespective of effect predictability. In this article, we review several classes of behavioral and neuroimaging data suggesting that earlier processes, linked to fluency of action selection, prospectively contribute to sense of agency. These findings have important implications for better understanding human volition and abnormalities of action experience.

Subliminal action priming modulates the perceived intensity of sensory action consequences.

The sense of control over the consequences of one's actions depends on predictions about these consequences. According to an influential computational model, consistency between predicted and observed action consequences attenuates perceived stimulus intensity, which might provide a marker of agentic control. An important assumption of this model is that these predictions are generated within the motor system. However, previous studies of sensory attenuation have typically confounded motor-specific perceptual modulation with perceptual effects of stimulus predictability that are not specific to motor action. As a result, these studies cannot unambiguously attribute sensory attenuation to a motor locus. We present a psychophysical experiment on auditory attenuation that avoids this pitfall. Subliminal masked priming of motor actions with compatible prime-target pairs has previously been shown to modulate both reaction times and the explicit feeling of control over action consequences. Here, we demonstrate reduced perceived loudness of tones caused by compatibly primed actions. Importantly, this modulation results from a manipulation of motor processing and is not confounded by stimulus predictability. We discuss our results with respect to theoretical models of the mechanisms underlying sensory attenuation and subliminal motor priming.