Effort Foraging Task reveals positive correlation between individual differences in the cost of cognitive and physical effort in humans.

Effort-based decisions, in which people weigh potential future rewards against effort costs required to achieve those rewards involve both cognitive and physical effort, though the mechanistic relationship between them is not yet understood. Here, we use an individual differences approach to isolate and measure the computational processes underlying effort-based decisions and test the association between cognitive and physical domains. Patch foraging is an ecologically valid reward rate maximization problem with well-developed theoretical tools. We developed the Effort Foraging Task, which embedded cognitive or physical effort into patch foraging, to quantify the cost of both cognitive and physical effort indirectly, by their effects on foraging choices. Participants chose between harvesting a depleting patch, or traveling to a new patch that was costly in time and effort. Participants' exit thresholds (reflecting the reward they expected to receive by harvesting when they chose to travel to a new patch) were sensitive to cognitive and physical effort demands, allowing us to quantify the perceived effort cost in monetary terms. The indirect sequential choice style revealed effort-seeking behavior in a minority of participants (preferring high over low effort) that has apparently been missed by many previous approaches. Individual differences in cognitive and physical effort costs were positively correlated, suggesting that these are perceived and processed in common. We used canonical correlation analysis to probe the relationship of task measures to self-reported affect and motivation, and found correlations of cognitive effort with anxiety, cognitive function, behavioral activation, and self-efficacy, but no similar correlations with physical effort.

Toward a Rational and Mechanistic Account of Mental Effort.

In spite of its familiar phenomenology, the mechanistic basis for mental effort remains poorly understood. Although most researchers agree that mental effort is aversive and stems from limitations in our capacity to exercise cognitive control, it is unclear what gives rise to those limitations and why they result in an experience of control as costly. The presence of these control costs also raises further questions regarding how best to allocate mental effort to minimize those costs and maximize the attendant benefits. This review explores recent advances in computational modeling and empirical research aimed at addressing these questions at the level of psychological process and neural mechanism, examining both the limitations to mental effort exertion and how we manage those limited cognitive resources. We conclude by identifying remaining challenges for theoretical accounts of mental effort as well as possible applications of the available findings to understanding the causes of and potential solutions for apparent failures to exert the mental effort required of us.

Make or Break: The Influence of Expected Challenges and Rewards on the Motivation and Experience Associated with Cognitive Effort Exertion.

Challenging goals can induce harder work but also greater stress, in turn potentially undermining goal achievement. We sought to examine how mental effort and subjective experiences thereof interact as a function of the challenge level and the size of the incentives at stake. Participants performed a task that rewarded individual units of effort investment (correctly performed Stroop trials) but only if they met a threshold number of correct trials within a fixed time interval (challenge level). We varied this challenge level (Study 1, n = 40) and the rewards at stake (Study 2, n = 79) and measured variability in task performance and self-reported affect across task intervals. Greater challenge and higher rewards facilitated greater effort investment but also induced greater stress, whereas higher rewards (and lower challenge) simultaneously induced greater positive affect. Within intervals, we observed an initial speed up then slowdown in performance, which could reflect dynamic reconfiguration of control. Collectively, these findings further our understanding of the influence of task demands and incentives on mental effort exertion and well-being.

Learning when effort matters: neural dynamics underlying updating and adaptation to changes in performance efficacy.

To determine how much cognitive control to invest in a task, people need to consider whether exerting control matters for obtaining rewards. In particular, they need to account for the efficacy of their performance-the degree to which rewards are determined by performance or by independent factors. Yet it remains unclear how people learn about their performance efficacy in an environment. Here we combined computational modeling with measures of task performance and EEG, to provide a mechanistic account of how people (i) learn and update efficacy expectations in a changing environment and (ii) proactively adjust control allocation based on current efficacy expectations. Across 2 studies, subjects performed an incentivized cognitive control task while their performance efficacy (the likelihood that rewards are performance-contingent or random) varied over time. We show that people update their efficacy beliefs based on prediction errors-leveraging similar neural and computational substrates as those that underpin reward learning-and adjust how much control they allocate according to these beliefs. Using computational modeling, we show that these control adjustments reflect changes in information processing, rather than the speed-accuracy tradeoff. These findings demonstrate the neurocomputational mechanism through which people learn how worthwhile their cognitive control is.

Rat Anterior Cingulate Cortex Continuously Signals Decision Variables in a Patch Foraging Task.

In patch foraging tasks, animals must decide whether to remain with a depleting resource or to leave it in search of a potentially better source of reward. In such tasks, animals consistently follow the general predictions of optimal foraging theory (the marginal value theorem; MVT): to leave a patch when the reward rate in the current patch depletes to the average reward rate across patches. Prior studies implicate an important role for the anterior cingulate cortex (ACC) in foraging decisions based on MVT: within single trials, ACC activity increases immediately preceding foraging decisions, and across trials, these dynamics are modulated as the value of staying in the patch depletes to the average reward rate. Here, we test whether these activity patterns reflect dynamic encoding of decision-variables and whether these signals are directly involved in decision-making. We developed a leaky accumulator model based on the MVT that generates estimates of decision variables within and across trials, and tested model predictions against ACC activity recorded from male rats performing a patch foraging task. Model predicted changes in MVT decision variables closely matched rat ACC activity. Next, we pharmacologically inactivated ACC in male rats to test the contribution of these signals to decision-making. ACC inactivation had a profound effect on rats' foraging decisions and response times (RTs) yet rats still followed the MVT decision rule. These findings indicate that the ACC encodes foraging-related variables for reasons unrelated to patch-leaving decisions.SIGNIFICANCE STATEMENT The ability to make adaptive patch-foraging decisions, to remain with a depleting resource or search for better alternatives, is critical to animal well-being. Previous studies have found that anterior cingulate cortex (ACC) activity is modulated at different points in the foraging decision process, raising questions about whether the ACC guides ongoing decisions or serves a more general purpose of regulating cognitive control. To investigate the function of the ACC in foraging, the present study developed a dynamic model of behavior and neural activity, and tested model predictions using recordings and inactivation of ACC. Findings revealed that ACC continuously signals decision variables but that these signals are more likely used to monitor and regulate ongoing processes than to guide foraging decisions.

Disentangling choice value and choice conflict in sequential decisions under risk.

Recent years have witnessed a surge of interest in understanding the neural and cognitive dynamics that drive sequential decision making in general and foraging behavior in particular. Due to the intrinsic properties of most sequential decision-making paradigms, however, previous research in this area has suffered from the difficulty to disentangle properties of the decision related to (a) the value of switching to a new patch versus, which increases monotonically, and (b) the conflict experienced between choosing to stay or leave, which first increases but then decreases after reaching the point of indifference between staying and switching. Here, we show how the same problems arise in studies of sequential decision-making under risk, and how they can be overcome, taking as a specific example recent research on the 'pig' dice game. In each round of the 'pig' dice game, people roll a die and accumulate rewards until they either decide to proceed to the next round or lose all rewards. By combining simulation-based dissections of the task structure with two experiments, we show how an extension of the standard paradigm, together with cognitive modeling of decision-making processes, allows to disentangle properties related to either switch value or choice conflict. Our study elucidates the cognitive mechanisms of sequential decision making and underscores the importance of avoiding potential pitfalls of paradigms that are commonly used in this research area.

Learning how to reason and deciding when to decide.

Research on human reasoning has both popularized and struggled with the idea that intuitive and deliberate thoughts stem from two different systems, raising the question how people switch between them. Inspired by research on cognitive control and conflict monitoring, we argue that detecting the need for further thought relies on an intuitive, context-sensitive process that is learned in itself.

Cognitive Control as a Multivariate Optimization Problem.

A hallmark of adaptation in humans and other animals is our ability to control how we think and behave across different settings. Research has characterized the various forms cognitive control can take-including enhancement of goal-relevant information, suppression of goal-irrelevant information, and overall inhibition of potential responses-and has identified computations and neural circuits that underpin this multitude of control types. Studies have also identified a wide range of situations that elicit adjustments in control allocation (e.g., those eliciting signals indicating an error or increased processing conflict), but the rules governing when a given situation will give rise to a given control adjustment remain poorly understood. Significant progress has recently been made on this front by casting the allocation of control as a decision-making problem. This approach has developed unifying and normative models that prescribe when and how a change in incentives and task demands will result in changes in a given form of control. Despite their successes, these models, and the experiments that have been developed to test them, have yet to face their greatest challenge: deciding how to select among the multiplicity of configurations that control can take at any given time. Here, we will lay out the complexities of the inverse problem inherent to cognitive control allocation, and their close parallels to inverse problems within motor control (e.g., choosing between redundant limb movements). We discuss existing solutions to motor control's inverse problems drawn from optimal control theory, which have proposed that effort costs act to regularize actions and transform motor planning into a well-posed problem. These same principles may help shed light on how our brains optimize over complex control configuration, while providing a new normative perspective on the origins of mental effort.

Dissociable influences of reward and punishment on adaptive cognitive control.

To invest effort into any cognitive task, people must be sufficiently motivated. Whereas prior research has focused primarily on how the cognitive control required to complete these tasks is motivated by the potential rewards for success, it is also known that control investment can be equally motivated by the potential negative consequence for failure. Previous theoretical and experimental work has yet to examine how positive and negative incentives differentially influence the manner and intensity with which people allocate control. Here, we develop and test a normative model of control allocation under conditions of varying positive and negative performance incentives. Our model predicts, and our empirical findings confirm, that rewards for success and punishment for failure should differentially influence adjustments to the evidence accumulation rate versus response threshold, respectively. This dissociation further enabled us to infer how motivated a given person was by the consequences of success versus failure.

Learning to Overexert Cognitive Control in a Stroop Task.

How do people learn when to allocate how much cognitive control to which task? According to the Learned Value of Control (LVOC) model, people learn to predict the value of alternative control allocations from features of a situation. This suggests that people may generalize the value of control learned in one situation to others with shared features, even when demands for control are different. This makes the intriguing prediction that what a person learned in one setting could cause them to misestimate the need for, and potentially overexert, control in another setting, even if this harms their performance. To test this prediction, we had participants perform a novel variant of the Stroop task in which, on each trial, they could choose to either name the color (more control-demanding) or read the word (more automatic). Only one of these tasks was rewarded each trial and could be predicted by one or more stimulus features (the color and/or word). Participants first learned colors and then words that predicted the rewarded task. Then, we tested how these learned feature associations transferred to novel stimuli with some overlapping features. The stimulus-task-reward associations were designed so that for certain combinations of stimuli, transfer of learned feature associations would incorrectly predict that more highly rewarded task would be color-naming, even though the actually rewarded task was word-reading and therefore did not require engaging control. Our results demonstrated that participants overexerted control for these stimuli, providing support for the feature-based learning mechanism described by the LVOC model.

Rational metareasoning and the plasticity of cognitive control.

The human brain has the impressive capacity to adapt how it processes information to high-level goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we develop and evaluate a model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert. We derive this model from a general theory according to which the function of cognitive control is to select and configure neural pathways so as to make optimal use of finite time and limited computational resources. The central idea of our Learned Value of Control model is that people use reinforcement learning to predict the value of candidate control signals of different types and intensities based on stimulus features. This model correctly predicts the learning and transfer effects underlying the adaptive control-demanding behavior observed in an experiment on visual attention and four experiments on interference control in Stroop and Flanker paradigms. Moreover, our model explained these findings significantly better than an associative learning model and a Win-Stay Lose-Shift model. Our findings elucidate how learning and experience might shape people's ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure.

A Control Theoretic Model of Adaptive Learning in Dynamic Environments.

To behave adaptively in environments that are noisy and nonstationary, humans and other animals must monitor feedback from their environment and adjust their predictions and actions accordingly. An understudied approach for modeling these adaptive processes comes from the engineering field of control theory, which provides general principles for regulating dynamical systems, often without requiring a generative model. The proportional-integral-derivative (PID) controller is one of the most popular models of industrial process control. The proportional term is analogous to the "delta rule" in psychology, adjusting estimates in proportion to each error in prediction. The integral and derivative terms augment this update to simultaneously improve accuracy and stability. Here, we tested whether the PID algorithm can describe how people sequentially adjust their predictions in response to new information. Across three experiments, we found that the PID controller was an effective model of participants' decisions in noisy, changing environments. In Experiment 1, we reanalyzed a change-point detection experiment and showed that participants' behavior incorporated elements of PID updating. In Experiments 2-3, we developed a task with gradual transitions that we optimized to detect PID-like adjustments. In both experiments, the PID model offered better descriptions of behavioral adjustments than both the classical delta-rule model and its more sophisticated variant, the Kalman filter. We further examined how participants weighted different PID terms in response to salient environmental events, finding that these control terms were modulated by reward, surprise, and outcome entropy. These experiments provide preliminary evidence that adaptive learning in dynamic environments resembles PID control.

Increased locus coeruleus tonic activity causes disengagement from a patch-foraging task.

High levels of locus coeruleus (LC) tonic activity are associated with distraction and poor performance within a task. Adaptive gain theory (AGT; Aston-Jones & Cohen, 2005) suggests that this may reflect an adaptive function of the LC, encouraging search for more remunerative opportunities in times of low utility. Here, we examine whether stimulating LC tonic activity using designer receptors (DREADDs) promotes searching for better opportunities in a patch-foraging task as the value of a patch diminishes. The task required rats to decide repeatedly whether to exploit an immediate but depleting reward within a patch or to incur the cost of a time delay to travel to a new, fuller patch. Similar to behavior associated with high LC tonic activity in other tasks, we found that stimulating LC tonic activity impaired task performance, resulting in reduced task participation and increased response times and omission rates. However, this was accompanied by a more specific, predicted effect: a significant tendency to leave patches earlier, which was best explained by an increase in decision noise rather than a systematic bias to leave earlier (i.e., at higher values). This effect is consistent with the hypothesis that high LC tonic activity favors disengagement from current behavior, and the pursuit of alternatives, by augmenting processing noise. These results provide direct causal evidence for the relationship between LC tonic activity and flexible task switching proposed by AGT.

Neural correlates of dueling affective reactions to win-win choices.

Win-win choices cause anxiety, often more so than decisions lacking the opportunity for a highly desired outcome. These anxious feelings can paradoxically co-occur with positive feelings, raising important implications for individual decision styles and general well-being. Across three studies, people chose between products that varied in personal value. Participants reported feeling most positive and most anxious when choosing between similarly high-valued products. Behavioral and neural results suggested that this paradoxical experience resulted from parallel evaluations of the expected outcome (inducing positive affect) versus the cost of choosing a response (inducing anxiety). Positive feelings were reduced when there was no high-value option, and anxiety was reduced when only one option was highly valued. Dissociable regions within the striatum and the medial prefrontal cortex (mPFC) tracked these dueling affective reactions during choice. Ventral regions, associated with stimulus valuation, tracked positive feelings and the value of the best item. Dorsal regions, associated with response valuation, tracked anxiety. In addition to tracking anxiety, the dorsal mPFC was associated with conflict during the current choice, and activity levels across individual items predicted whether that choice would later be reversed during an unexpected reevaluation phase. By revealing how win-win decisions elicit responses in dissociable brain systems, these results help resolve the paradox of win-win choices. They also provide insight into behaviors that are associated with these two forms of affect, such as why we are pulled toward good options but may still decide to delay or avoid choosing among them.

Dorsal anterior cingulate and ventromedial prefrontal cortex have inverse roles in both foraging and economic choice.

Recent research has highlighted a distinction between sequential foraging choices and traditional economic choices between simultaneously presented options. This was partly motivated by observations in Kolling, Behrens, Mars, and Rushworth, Science, 336(6077), 95-98 (2012) (hereafter, KBMR) that these choice types are subserved by different circuits, with dorsal anterior cingulate (dACC) preferentially involved in foraging and ventromedial prefrontal cortex (vmPFC) preferentially involved in economic choice. To support this account, KBMR used fMRI to scan human subjects making either a foraging choice (between exploiting a current offer or swapping for potentially better rewards) or an economic choice (between two reward-probability pairs). This study found that dACC better tracked values pertaining to foraging, whereas vmPFC better tracked values pertaining to economic choice. We recently showed that dACC's role in these foraging choices is better described by the difficulty of choosing than by foraging value, when correcting for choice biases and testing a sufficiently broad set of foraging values (Shenhav, Straccia, Cohen, & Botvinick Nature Neuroscience, 17(9), 1249-1254, 2014). Here, we extend these findings in 3 ways. First, we replicate our original finding with a larger sample and a task modified to address remaining methodological gaps between our previous experiments and that of KBMR. Second, we show that dACC activity is best accounted for by choice difficulty alone (rather than in combination with foraging value) during both foraging and economic choices. Third, we show that patterns of vmPFC activity, inverted relative to dACC, also suggest a common function across both choice types. Overall, we conclude that both regions are similarly engaged by foraging-like and economic choice.

Integrative moral judgment: dissociating the roles of the amygdala and ventromedial prefrontal cortex.

A decade's research highlights a critical dissociation between automatic and controlled influences on moral judgment, which is subserved by distinct neural structures. Specifically, negative automatic emotional responses to prototypically harmful actions (e.g., pushing someone off of a footbridge) compete with controlled responses favoring the best consequences (e.g., saving five lives instead of one). It is unknown how such competitions are resolved to yield "all things considered" judgments. Here, we examine such integrative moral judgments. Drawing on insights from research on self-interested, value-based decision-making in humans and animals, we test a theory concerning the respective contributions of the amygdala and ventromedial prefrontal cortex (vmPFC) to moral judgment. Participants undergoing fMRI responded to moral dilemmas, separately evaluating options for their utility (Which does the most good?), emotional aversiveness (Which feels worse?), and overall moral acceptability. Behavioral data indicate that emotional aversiveness and utility jointly predict "all things considered" integrative judgments. Amygdala response tracks the emotional aversiveness of harmful utilitarian actions and overall disapproval of such actions. During such integrative moral judgments, the vmPFC is preferentially engaged relative to utilitarian and emotional assessments. Amygdala-vmPFC connectivity varies with the role played by emotional input in the task, being the lowest for pure utilitarian assessments and the highest for pure emotional assessments. These findings, which parallel those of research on self-interested economic decision-making, support the hypothesis that the amygdala provides an affective assessment of the action in question, whereas the vmPFC integrates that signal with a utilitarian assessment of expected outcomes to yield "all things considered" moral judgments.

Human preferences are biased towards associative information.

There is ample evidence that the brain generates predictions that help interpret sensory input. To build such predictions the brain capitalizes upon learned statistical regularities and associations (e.g., "A" is followed by "B"; "C" appears together with "D"). The centrality of predictions to mental activities gave rise to the hypothesis that associative information with predictive value is perceived as intrinsically valuable. Such value would ensure that this information is proactively searched for, thereby promoting certainty and stability in our environment. We therefore tested here whether, all else being equal, participants would prefer stimuli that contained more rather than less associative information. In Experiments 1 and 2 we used novel, meaningless visual shapes and showed that participants preferred associative shapes over shapes that had not been associated with other shapes during training. In Experiment 3 we used pictures of real-world objects and again demonstrated a preference for stimuli that elicit stronger associations. These results support our proposal that predictive information is affectively tagged, and enhance our understanding of the formation of everyday preferences.

The affective gradient hypothesis: an affect-centered account of motivated behavior.

Everyone agrees that feelings and actions are intertwined, but cannot agree how. According to dominant models, actions are directed by estimates of value and these values shape or are shaped by affect. I propose instead that affect is the only form of value that drives actions. Our mind constantly represents potential future states and how they would make us feel. These states collectively form a gradient reflecting feelings we could experience depending on actions we take. Motivated behavior reflects the process of traversing this affective gradient, towards desirable states and away from undesirable ones. This affective gradient hypothesis solves the puzzle of where values and goals come from, and offers a parsimonious account of apparent conflicts between emotion and cognition.

Humans reconfigure target and distractor processing to address distinct task demands.

When faced with distraction, we can focus more on goal-relevant information (targets) or focus less on goal-conflicting information (distractors). How people use cognitive control to distribute attention across targets and distractors remains unclear. We address this question by developing a novel Parametric Attentional Control Task that can "tag" participants' sensitivity to target and distractor information. We use these precise measures of attention to develop a novel process model that can explain how participants control attention toward targets and distractors. Across three experiments, we find that participants met the demands of this task by independently controlling their processing of target and distractor information, exhibiting distinct adaptations to manipulations of incentives and conflict. Whereas incentives preferentially led to target enhancement, conflict in the previous trial preferentially led to distractor suppression. These distinct drivers of control altered sensitivity to targets and distractors early in the trial, promptly followed by reactive reconfiguration toward task-appropriate feature sensitivity. To provide a process-level account of these empirical findings, we develop a novel neural network model of evidence accumulation with attractor dynamics over feature weights that reconfigure target and distractor processing. These results provide a computational account of control reconfiguration that provides new insights into how multivariate attentional signals are optimized to achieve task goals. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Orthogonal neural encoding of targets and distractors supports multivariate cognitive control.

The complex challenges of our mental life require us to coordinate multiple forms of neural information processing. Recent behavioural studies have found that people can coordinate multiple forms of attention, but the underlying neural control process remains obscure. We hypothesized that the brain implements multivariate control by independently monitoring feature-specific difficulty and independently prioritizing feature-specific processing. During functional MRI, participants performed a parametric conflict task that separately tags target and distractor processing. Consistent with feature-specific monitoring, univariate analyses revealed spatially segregated encoding of target and distractor difficulty in the dorsal anterior cingulate cortex. Consistent with feature-specific attentional priority, our encoding geometry analysis revealed overlapping but orthogonal representations of target and distractor coherence in the intraparietal sulcus. Coherence representations were mediated by control demands and aligned with both performance and frontoparietal activity, consistent with top-down attention. Together, these findings provide evidence for the neural geometry necessary to coordinate multivariate cognitive control.