Atomoxetine and reward size equally improve task engagement and perceptual decisions but differently affect movement execution.

Our ability to engage and perform daily activities relies on balancing the associated benefits and costs. Rewards, as benefits, act as powerful motivators that help us stay focused for longer durations. The noradrenergic (NA) system is thought to play a significant role in optimizing our performance. Yet, the interplay between reward and the NA system in shaping performance remains unclear, particularly when actions are driven by external incentives (reward). To explore this interaction, we tested four female rhesus monkeys performing a sustained Go/NoGo task under two reward sizes (low/high) and three pharmacological conditions (saline and two doses of atomoxetine, a NA reuptake inhibitor: ATX-0.5 mg/kg and ATX-1 mg/kg). We found that increasing either reward or NA levels equally enhanced the animal's engagement in the task compared to low reward saline; the animals also responded faster and more consistently under these circumstances. Notably, we identified differences between reward size and ATX. When combined with ATX, high reward further reduced the occurrence of false alarms (i.e., incorrect go trials on distractors), implying that it helped further suppress impulsive responses. In addition, ATX (but not reward size) consistently increased movement duration dose-dependently, while high reward did not affect movement duration but decreased its variability. We conclude that noradrenaline and reward modulate performance, but their effects are not identical, suggesting differential underlying mechanisms. Reward might energize/invigorate decisions and action, while ATX might help regulate energy expenditure, depending on the context, through the NA system.

Integrated control of non-motor and motor efforts during perceptual decision-making and action execution: a pilot study.

Humans daily life is characterized by a succession of voluntary actions. Since energy resources are limited, the ability to invest the appropriate amount of effort for selecting and executing these actions is a hallmark of adapted behavior. Recent studies indicate that decisions and actions share important principles, including the optimization of their duration when the context requires it. In the present pilot study, we test the hypothesis that the management of effort-related energy resources is shared between decision and action too. Healthy human subjects performed a perceptual decision task where they had to choose between two levels of effort to invest in making the decision (i.e. two levels of perceptual difficulty), and report it with a reaching movement. Crucially, the movement accuracy requirement gradually increased from trial to trial depending on participants' decision performance. Results indicate an overall moderate and non-significant impact of the increasing motor difficulty on the choice of the non-motor (decision) effort to invest in each trial and on decision performance. By contrast, motor performance strongly decreased depending on both the motor and decisional difficulties. Together, the results support the hypothesis of an integrated management of the effort-related energy resources between decision and action. They also suggest that in the present task, the mutualized resources are primarily allocated to the decision-making process to the detriment of movements.

Impact of decision and action outcomes on subsequent decision and action behaviours in humans.

Speed-accuracy trade-off adjustments in decision-making have been mainly studied separately from those in motor control. In the wild, however, animals coordinate their decision and action, often deciding while acting. Recent behavioural studies support this view, indicating that animals, including humans, trade decision time for movement time to maximize their global rate of reward during experimental sessions. Besides, it is well established that choice outcomes impact subsequent decisions. Crucially though, whether and how a decision outcome also influences the subsequent motor performance, and whether and how the outcome of a movement influences the next decision, is unclear. Here, we address these questions by analysing trial-to-trial changes of choice and motor behaviours in healthy human participants instructed to perform successive perceptual decisions expressed with reaching movements whose duration was either weakly or strongly constrained in separate tasks. Results indicate that after a wrong decision, subjects who were weakly constrained in their action duration decided more slowly and more accurately. Interestingly, they also shortened their subsequent movement duration by moving faster. Conversely, we found that errors of constrained movements influenced not only the speed and the amplitude of the following movement but those of the decision too. If the movement had to be slowed down, the decision that precedes that movement was accelerated and vice versa. Together, these results indicate that from one trial to the next, humans seek to determine a behavioural duration as a whole instead of optimizing each of the decision and action speed-accuracy trade-offs independently of each other.

Integrated neural dynamics of sensorimotor decisions and actions.

Recent theoretical models suggest that deciding about actions and executing them are not implemented by completely distinct neural mechanisms but are instead two modes of an integrated dynamical system. Here, we investigate this proposal by examining how neural activity unfolds during a dynamic decision-making task within the high-dimensional space defined by the activity of cells in monkey dorsal premotor (PMd), primary motor (M1), and dorsolateral prefrontal cortex (dlPFC) as well as the external and internal segments of the globus pallidus (GPe, GPi). Dimensionality reduction shows that the four strongest components of neural activity are functionally interpretable, reflecting a state transition between deliberation and commitment, the transformation of sensory evidence into a choice, and the baseline and slope of the rising urgency to decide. Analysis of the contribution of each population to these components shows meaningful differences between regions but no distinct clusters within each region, consistent with an integrated dynamical system. During deliberation, cortical activity unfolds on a two-dimensional "decision manifold" defined by sensory evidence and urgency and falls off this manifold at the moment of commitment into a choice-dependent trajectory leading to movement initiation. The structure of the manifold varies between regions: In PMd, it is curved; in M1, it is nearly perfectly flat; and in dlPFC, it is almost entirely confined to the sensory evidence dimension. In contrast, pallidal activity during deliberation is primarily defined by urgency. We suggest that these findings reveal the distinct functional contributions of different brain regions to an integrated dynamical system governing action selection and execution.

Hasty sensorimotor decisions rely on an overlap of broad and selective changes in motor activity.

Humans and other animals are able to adjust their speed-accuracy trade-off (SAT) at will depending on the urge to act, favoring either cautious or hasty decision policies in different contexts. An emerging view is that SAT regulation relies on influences exerting broad changes on the motor system, tuning its activity up globally when hastiness is at premium. The present study aimed to test this hypothesis. A total of 50 participants performed a task involving choices between left and right index fingers, in which incorrect choices led either to a high or to a low penalty in 2 contexts, inciting them to emphasize either cautious or hasty policies. We applied transcranial magnetic stimulation (TMS) on multiple motor representations, eliciting motor-evoked potentials (MEPs) in 9 finger and leg muscles. MEP amplitudes allowed us to probe activity changes in the corresponding finger and leg representations, while participants were deliberating about which index to choose. Our data indicate that hastiness entails a broad amplification of motor activity, although this amplification was limited to the chosen side. On top of this effect, we identified a local suppression of motor activity, surrounding the chosen index representation. Hence, a decision policy favoring speed over accuracy appears to rely on overlapping processes producing a broad (but not global) amplification and a surround suppression of motor activity. The latter effect may help to increase the signal-to-noise ratio of the chosen representation, as supported by single-trial correlation analyses indicating a stronger differentiation of activity changes in finger representations in the hasty context.

Dissociating the Impact of Movement Time and Energy Costs on Decision-Making and Action Initiation in Humans.

Recent theories and data suggest that adapted behavior involves economic computations during which multiple trade-offs between reward value, accuracy requirement, energy expenditure, and elapsing time are solved so as to obtain rewards as soon as possible while spending the least possible amount of energy. However, the relative impact of movement energy and duration costs on perceptual decision-making and movement initiation is poorly understood. Here, we tested 31 healthy subjects on a perceptual decision-making task in which they executed reaching movements to report probabilistic choices. In distinct blocks of trials, the reaching duration ("Time" condition) and energy ("Effort" condition) costs were independently varied compared to a "Reference" block, while decision difficulty was maintained similar at the block level. Participants also performed a simple delayed-reaching (DR) task aimed at estimating movement initiation duration in each motor condition. Results in that DR task show that long duration movements extended reaction times (RTs) in most subjects, whereas energy-consuming movements led to mixed effects on RTs. In the decision task, about half of the subjects decreased their decision durations (DDs) in the Time condition, while the impact of energy on DDs were again mixed across subjects. Decision accuracy was overall similar across motor conditions. These results indicate that movement duration and, to a lesser extent, energy expenditure, idiosyncratically affect perceptual decision-making and action initiation. We propose that subjects who shortened their choices in the time-consuming condition of the decision task did so to limit a drop of reward rate.

Reducing behavioral dimensions to study brain-environment interactions.

Movement vigor provides a window on action valuation. But what is vigor, and how to measure it in the first place? Strikingly, many different co-varying vigor-related metrics can be found in the literature. I believe this is because vigor, just like the neural circuits that determine it, is an integrated, low-dimensional parameter. As such, it can only be roughly estimated.

Trading accuracy for speed over the course of a decision.

Humans and other animals often need to balance the desire to gather sensory information (to make the best choice) with the urgency to act, facing a speed-accuracy tradeoff (SAT). Given the ubiquity of SAT across species, extensive research has been devoted to understanding the computational mechanisms allowing its regulation at different timescales, including from one context to another, and from one decision to another. However, animals must frequently change their SAT on even shorter timescales-that is, over the course of an ongoing decision-and little is known about the mechanisms that allow such rapid adaptations. The present study aimed at addressing this issue. Human subjects performed a decision task with changing evidence. In this task, subjects received rewards for correct answers but incurred penalties for mistakes. An increase or a decrease in penalty occurring halfway through the trial promoted rapid SAT shifts, favoring speeded decisions either in the early or in the late stage of the trial. Importantly, these shifts were associated with stage-specific adjustments in the accuracy criterion exploited for committing to a choice. Those subjects who decreased the most their accuracy criterion at a given decision stage exhibited the highest gain in speed, but also the highest cost in terms of performance accuracy at that time. Altogether, the current findings offer a unique extension of previous work, by suggesting that dynamic cha*nges in accuracy criterion allow the regulation of the SAT within the timescale of a single decision.NEW & NOTEWORTHY Extensive research has been devoted to understanding the mechanisms allowing the regulation of the speed-accuracy tradeoff (SAT) from one context to another and from one decision to another. Here, we show that humans can voluntarily change their SAT on even shorter timescales-that is, over the course of a decision. These rapid SAT shifts are associated with dynamic adjustments in the accuracy criterion exploited for committing to a choice.

Humans sacrifice decision-making for action execution when a demanding control of movement is required.

A growing body of evidence suggests that decision-making and action execution are governed by partly overlapping operating principles. Especially, previous work proposed that a shared decision urgency/movement vigor signal, possibly computed in the basal ganglia, coordinates both deliberation and movement durations in a way that maximizes the reward rate. Recent data support one aspect of this hypothesis, indicating that the urgency level at which a decision is made influences the vigor of the movement produced to express this choice. Here we investigated whether, conversely, the motor context in which a movement is executed determines decision speed and accuracy. Twenty human subjects performed a probabilistic decision task in which perceptual choices were expressed by reaching movements toward targets whose size and distance from a starting position varied in distinct blocks of trials. We found strong evidence for an influence of the motor context on most of the subjects' decision policy, but contrary to the predictions of the "shared regulation" hypothesis, we observed that slow movements executed in the most demanding motor blocks in terms of accuracy were often preceded by faster and less accurate decisions compared with blocks of trials in which big targets allowed expression of choices with fast and inaccurate movements. These results suggest that decision-making and motor control are not regulated by one unique "invigoration" signal determining both decision urgency and action vigor, but more likely by independent, yet interacting, decision urgency and movement vigor signals.NEW & NOTEWORTHY Recent hypotheses propose that choices and movements share optimization principles derived from economy, possibly implemented by one unique context-dependent regulation signal determining both processes' speed. In the present behavioral study conducted on human subjects, we demonstrate that action properties indeed influence perceptual decision-making, but that decision duration and action vigor are actually independently set depending on the difficulty of the movement executed to report a choice.

Decision urgency invigorates movement in humans.

Recent work in highly trained monkeys suggests that decision-making and motor control are linked processes whose regulation by urgency allows reward rate optimization. However, such urgency-based mechanism might be species-specific and/or a consequence of practice. Here I show that the unified regulation hypothesis exists in naïve human subjects. Seventeen volunteers performed a reach decision task in which blocks of trials encouraged either risky or conservative choices. Participants performed at least two sessions in which they were indirectly motivated to maximize their reward rate. Results show that subjects' accuracy criterion decreased over time within each trial, and that decisions were earlier and less accurate in fast than slow blocks, with a larger difference in session #2 compared to session #1. A simple model in which sensory information is combined with a growing urgency signal captured these effects. Crucially, arm and eye movement vigor systematically increased as decision duration increased within blocks and the block-dependent decision policy strongly predicted the kinematics of reaching movements. These results suggest a practice-independent mechanistic link for establishing the unified control of human decision timing and action vigor in order to optimize the rate of reward.

Microstimulation of dorsal premotor and primary motor cortex delays the volitional commitment to an action choice.

Humans and other animals are faced with decisions about actions on a daily basis. These typically include a period of deliberation that ends with the commitment to a choice, which then leads to the overt expression of that choice through action. Previous studies with monkeys have demonstrated that neural activity in sensorimotor areas correlates with the deliberation process and reflects the moment of commitment before movement initiation, but the causal roles of these regions are challenging to establish. Here, we tested whether dorsal premotor (PMd) and primary motor cortex (M1) are causally involved in the volitional commitment to a reaching choice. We found that brief subthreshold microstimulation in PMd or M1 delayed commitment to an action but not the initiation of the action itself. Importantly, microstimulation only had a significant effect when it was delivered close to and before commitment time. These results are consistent with the proposal that PMd and M1 participate in the commitment process, which occurs when a critical firing rate difference is reached between cells voting for the selected option and those voting for the competing one.NEW & NOTEWORTHY The neural substrates of decisions between actions are typically investigated by correlating neural activity and subjects' decision behavior, but this does not establish causality. In a reaching decision task, we demonstrate that subthreshold microstimulation of the monkey dorsal premotor cortex or primary motor cortex delays the deliberation duration if applied shortly before choice commitment. This result suggests a causal role of the sensorimotor cortex in the determination of decisions between actions.

Motor cortex disruption delays motor processes but not deliberation about action choices.

Decisions about actions typically involve a period of deliberation that ends with the commitment to a choice and the motor processes overtly expressing that choice. Previous studies have shown that neural activity in sensorimotor areas, including the primary motor cortex (M1), correlates with deliberation features during action selection. However, the causal contribution of these areas to the decision process remains unclear. Here, we investigated whether M1 determines choice commitment or whether it simply reflects decision signals coming from upstream structures and instead mainly contributes to the motor processes that follow commitment. To do so, we tested the impact of a disruption of M1 activity, induced by continuous theta burst stimulation (cTBS), on the behavior of human subjects in 1) a simple reaction time (SRT) task allowing us to estimate the duration of the motor processes and 2) a modified version of the tokens task (Cisek P, Puskas GA, El-Murr S. J Neurosci 29: 11560-11571, 2009), which allowed us to estimate subjects' time of commitment as well as accuracy criterion. The efficiency of cTBS was attested by a reduction in motor evoked potential amplitudes following M1 disruption compared with those following a sham stimulation. Furthermore, M1 cTBS lengthened SRTs, indicating that motor processes were perturbed by the intervention. Importantly, all of the behavioral results in the tokens task were similar following M1 disruption and sham stimulation, suggesting that the contribution of M1 to the deliberation process is potentially negligible. Taken together, these findings favor the view that M1 contribution is downstream of the decision process.NEW & NOTEWORTHY Decisions between actions are ubiquitous in the animal realm. Deliberation during action choices entails changes in the activity of the sensorimotor areas controlling those actions, but the causal role of these areas is still often debated. With the use of continuous theta burst stimulation, we show that disrupting the primary motor cortex (M1) delays the motor processes that follow instructed commitment but does not alter volitional deliberation, suggesting that M1 contribution may be downstream of the decision process.

Thirst for Intention? Grasping a Glass Is a Thirst-Controlled Action.

Every day and every hour, we feel we perform numerous voluntary actions, i.e., actions under the control of our will. Individual's ability to initiate goal-directed movement is classically described as a hierarchical motor organization, from an intentional module, mostly considered as a black box, to muscular activity supporting action execution. The general focus is usually set on the triggering of action by intention, which is assumed to be the only entry to the action cascade, rather than on the preceding formation of intentions. If intentions play a key role in the specification of movement kinematic parameters, it remains largely unknown whether unconscious cognitive processes may also affect action preparation and unfolding. Recently, a seemingly irrelevant variable, thirst, was shown to modulate a simple arbitrary action such as key-pressing. Thirsty individuals were shown to produce stronger motor inhibition in no-go trials when a glass of water was present. In the present experiment, we intended to explore whether motor inhibition operates not only upstream from the action cascade but may also affect the unfolding of reaching movements, i.e., at a lower-level control. Thirsty vs. non-thirsty control subjects were asked to reach and grasp green (go trial) or red glasses (no-go trial) filled with either water or transparent gel wax with a central candlewick. Thirsty subjects were faster to initiate actions toward the water glasses. They also exhibited an earlier maximal grip aperture and a global reduction of movement time which was mostly explained by a shortening of deceleration time. The deceleration phase was correlated with individual's thirst rating. In addition, no-go trial toward a glass of water tended to inhibit the next movement toward a glass filled with gel wax. Thus, our results show that an unintentional influence of an internal state can reorganize voluntary action structure not only at the decision-making level but also at the level of motor control. Although subjects explicitly paid more attention and were more cautious to glasses filled with water, they reported no explicit sensation of an increased urge to grasp it, further suggesting that these effects are controlled by covert mechanisms.

The Urge to Decide and Act: Implications for Brain Function and Dysfunction.

Humans and other animals are motivated to act so as to maximize their subjective reward rate. Here, we propose that reward rate maximization is accomplished by adjusting a context-dependent "urgency signal," which influences both the commitment to a developing action choice and the vigor with which the ensuing action is performed. We review behavioral and neurophysiological data suggesting that urgency is controlled by projections from the basal ganglia to cerebral cortical regions, influencing neural activity related to decision making as well as activity related to action execution. We also review evidence suggesting that different individuals possess specific policies for adjusting their urgency signal to particular contextual variables, such that urgency constitutes an individual trait which jointly influences a wide range of behavioral measures commonly related to the overall quality and hastiness of one's decisions and actions. Consequently, we argue that a central mechanism for reward rate maximization provides a potential link between personality traits such as impulsivity, as well as some of the motivation-related symptomology of clinical disorders such as depression and Parkinson's disease.

The Basal Ganglia Do Not Select Reach Targets but Control the Urgency of Commitment.

Prominent theories of decision making suggest that the basal ganglia (BG) play a causal role in deliberation between action choices. An alternative hypothesis is that deliberation occurs in cortical regions, while the BG control the speed-accuracy trade-off (SAT) between committing to a choice versus continuing to deliberate. Here, we test these hypotheses by recording activity in the internal and external segments of the globus pallidus (GPi/GPe) while monkeys perform a task dissociating the process of deliberation, the moment of commitment, and adjustment of the SAT. Our data suggest that unlike premotor and motor cortical regions, pallidal output does not contribute to the process of deliberation but instead provides a time-varying signal that controls the SAT and reflects the growing urgency to commit to a choice. Once a target is selected by cortical regions, GP activity confirms commitment to the decision and invigorates the subsequent movement.

Trial-to-trial adjustments of speed-accuracy trade-offs in premotor and primary motor cortex.

Recent studies have shown that activity in sensorimotor structures varies depending on the speed-accuracy trade-off (SAT) context in which a decision is made. Here we tested the hypothesis that the same areas also reflect a more local adjustment of SAT established between individual trials, based on the outcome of the previous decision. Two monkeys performed a reaching decision task in which sensory evidence continuously evolves during the time course of a trial. In two SAT contexts, we compared neural activity in trials following a correct choice vs. those following an error. In dorsal premotor cortex (PMd), we found that 23% of cells exhibited significantly weaker baseline activity after error trials, and for ∼30% of these this effect persisted into the deliberation epoch. These cells also contributed to the process of combining sensory evidence with the growing urgency to commit to a choice. We also found that the activity of 22% of PMd cells was increased after error trials. These neurons appeared to carry less information about sensory evidence and time-dependent urgency. For most of these modulated cells, the effect was independent of whether the previous error was expected or unexpected. We found similar phenomena in primary motor cortex (M1), with 25% of cells decreasing and 34% increasing activity after error trials, but unlike PMd, these neurons showed less clear differences in their response properties. These findings suggest that PMd and M1 belong to a network of brain areas involved in SAT adjustments established using the recent history of reinforcement. NEW & NOTEWORTHY: Setting the speed-accuracy trade-off (SAT) is crucial for efficient decision making. Previous studies have reported that subjects adjust their SAT after individual decisions, usually choosing more conservatively after errors, but the neural correlates of this phenomenon are only partially known. Here, we show that neurons in PMd and M1 of monkeys performing a reach decision task support this mechanism by adequately modulating their firing rate as a function of the outcome of the previous decision.

Modulation of Premotor and Primary Motor Cortical Activity during Volitional Adjustments of Speed-Accuracy Trade-Offs.

Recent work suggests that while animals decide between reaching actions, neurons in dorsal premotor (PMd) and primary motor (M1) cortex reflect a dynamic competition between motor plans and determine when commitment to a choice is made. This competition is biased by at least two sources of information: the changing sensory evidence for one choice versus another, and an urgency signal that grows over time. Here, we test the hypothesis that the urgency signal adjusts the trade-off between speed and accuracy during both decision-making and movement execution. Two monkeys performed a reaching decision task in which sensory evidence continuously evolves over the course of each trial. In different blocks, task timing parameters encouraged monkeys to voluntarily adapt their behavior to be either hasty or conservative. Consistent with our hypothesis, during the deliberation process the baseline and gain of neural activity in decision-related PMd (29%) and M1 cells (45%) was higher when monkeys applied a hasty policy than when they behaved conservatively, but at the time of commitment the population activity was similar across blocks. Other cells (30% in PMd, 30% in M1) showed activity that increased or decreased with elapsing time until the moment of commitment. Movement-related neurons were also more active after longer decisions, as if they were influenced by the same urgency signal controlling the gain of decision-related activity. Together, these results suggest that the arm motor system receives an urgency/vigor signal that adjusts the speed-accuracy trade-off for decision-making and movement execution. Significance statement: This work addresses the neural mechanisms that control the speed-accuracy trade-off in both decisions and movements, in the kinds of dynamic situations that are typical of natural animal behavior. We found that many "decision-related" premotor and motor neurons are modulated in a time-dependent manner compatible with an "urgency" signal that changes between hasty and conservative decision policies. We also found that such modulation influenced cells related to the speed of the reaching movements executed by the animals to report their decisions. These results suggest that a unified mechanism determines speed-accuracy trade-off adjustments during decision-making and movement execution, potentially influencing both the cognitive and motor aspects of reward-oriented behavior.

How to discriminate conclusively among different models of decision making?

A popular and successful class of decision-making models (the "evidence accumulator" models) has been recently challenged by a new hypothesis called the urgency-gating model. Hawkins et al. (J Neurophysiol 114: 40-47, 2015) used a sophisticated curve-fitting procedure to show that these models are discriminable and thus testable in constant evidence tasks. In this Neuro Forum article I raise possible limitations of such an approach, discuss some of its implications, and propose alternative solutions.