Dissociation and integration of outcome and state uncertainty signals in cognitive control.

Signals related to uncertainty are frequently observed in regions of the cognitive control network, including anterior cingulate/medial prefrontal cortex (ACC/mPFC), dorsolateral prefrontal cortex (dlPFC), and anterior insular cortex. Uncertainty generally refers to conditions in which decision variables may assume multiple possible values and can arise at multiple points in the perception-action cycle, including sensory input, inferred states of the environment, and the consequences of actions. These sources of uncertainty are frequently correlated: noisy input can lead to unreliable estimates of the state of the environment, with consequential influences on action selection. Given this correlation amongst various sources of uncertainty, dissociating the neural structures underlying their estimation presents an ongoing issue: a region associated with uncertainty related to outcomes may estimate outcome uncertainty itself, or it may reflect a cascade effect of state uncertainty on outcome estimates. In this study, we derive signals of state and outcome uncertainty from mathematical models of risk and observe regions in the cognitive control network whose activity is best explained by signals related to state uncertainty (anterior insula), outcome uncertainty (dlPFC), as well as regions that appear to integrate the two (ACC/mPFC).

More than a feeling: physiological measures of affect index the integration of effort costs and rewards during anticipatory effort evaluation.

The notion that humans avoid effortful action is one of the oldest and most persistent in psychology. Influential theories of effort propose that effort valuations are made according to a cost-benefit trade-off: we tend to invest mental effort only when the benefits outweigh the costs. While these models provide a useful conceptual framework, the affective components of effort valuation remain poorly understood. Here, we examined whether primitive components of affective response-positive and negative valence, captured via facial electromyography (fEMG)-can be used to better understand valuations of cognitive effort. Using an effortful arithmetic task, we find that fEMG activity in the corrugator supercilii-thought to index negative valence-1) tracks the anticipation and exertion of cognitive effort and 2) is attenuated in the presence of high rewards. Together, these results suggest that activity in the corrugator reflects the integration of effort costs and rewards during effortful decision-making.

Detecting Prolonged Stress in Real Life Using Wearable Biosensors and Ecological Momentary Assessments: Naturalistic Experimental Study.

BACKGROUND: Increasing efforts toward the prevention of stress-related mental disorders have created a need for unobtrusive real-life monitoring of stress-related symptoms. Wearable devices have emerged as a possible solution to aid in this process, but their use in real-life stress detection has not been systematically investigated. OBJECTIVE: We aimed to determine the utility of ecological momentary assessments (EMA) and physiological arousal measured through wearable devices in detecting ecologically relevant stress states. METHODS: Using EMA combined with wearable biosensors for ecological physiological assessments (EPA), we investigated the impact of an ecological stressor (ie, a high-stakes examination week) on physiological arousal and affect compared to a control week without examinations in first-year medical and biomedical science students (51/83, 61.4% female). We first used generalized linear mixed-effects models with maximal fitting approaches to investigate the impact of examination periods on subjective stress exposure, mood, and physiological arousal. We then used machine learning models to investigate whether we could use EMA, wearable biosensors, or the combination of both to classify momentary data (ie, beeps) as belonging to examination or control weeks. We tested both individualized models using a leave-one-beep-out approach and group-based models using a leave-one-subject-out approach. RESULTS: During stressful high-stakes examination (versus control) weeks, participants reported increased negative affect and decreased positive affect. Intriguingly, physiological arousal decreased on average during the examination week. Time-resolved analyses revealed peaks in physiological arousal associated with both momentary self-reported stress exposure and self-reported positive affect. Mediation models revealed that the decreased physiological arousal in the examination week was mediated by lower positive affect during the same period. We then used machine learning to show that while individualized EMA outperformed EPA in its ability to classify beeps as originating from examinations or from control weeks (1603/4793, 33.45% and 1648/4565, 36.11% error rates, respectively), a combination of EMA and EPA yields optimal classification (1363/4565, 29.87% error rate). Finally, when comparing individualized models to group-based models, we found that the individualized models significantly outperformed the group-based models across all 3 inputs (EMA, EPA, and the combination). CONCLUSIONS: This study underscores the potential of wearable biosensors for stress-related mental health monitoring. However, it emphasizes the necessity of psychological context in interpreting physiological arousal captured by these devices, as arousal can be related to both positive and negative contexts. Moreover, our findings support a personalized approach in which momentary stress is optimally detected when referenced against an individual's own data.

Challenging the negative learning bias hypothesis of depression: reversal learning in a naturalistic psychiatric sample.

BACKGROUND: Classic theories posit that depression is driven by a negative learning bias. Most studies supporting this proposition used small and selected samples, excluding patients with comorbidities. However, comorbidity between psychiatric disorders occurs in up to 70% of the population. Therefore, the generalizability of the negative bias hypothesis to a naturalistic psychiatric sample as well as the specificity of the bias to depression, remain unclear. In the present study, we tested the negative learning bias hypothesis in a large naturalistic sample of psychiatric patients, including depression, anxiety, addiction, attention-deficit/hyperactivity disorder, and/or autism. First, we assessed whether the negative bias hypothesis of depression generalized to a heterogeneous (and hence more naturalistic) depression sample compared with controls. Second, we assessed whether negative bias extends to other psychiatric disorders. Third, we adopted a dimensional approach, by using symptom severity as a way to assess associations across the sample. METHODS: We administered a probabilistic reversal learning task to 217 patients and 81 healthy controls. According to the negative bias hypothesis, participants with depression should exhibit enhanced learning and flexibility based on punishment v. reward. We combined analyses of traditional measures with more sensitive computational modeling. RESULTS: In contrast to previous findings, this sample of depressed patients with psychiatric comorbidities did not show a negative learning bias. CONCLUSIONS: These results speak against the generalizability of the negative learning bias hypothesis to depressed patients with comorbidities. This study highlights the importance of investigating unselected samples of psychiatric patients, which represent the vast majority of the psychiatric population.

Task-specific prioritization of reward and effort information: Novel insights from behavior and computational modeling.

Efficient integration of environmental information is critical in goal-directed behavior. Motivational information regarding potential rewards and costs (such as required effort) affects performance and decisions whether to engage in a task. While it is generally acknowledged that costs and benefits are integrated to determine the level of effort to be exerted, how this integration occurs remains an open question. Computational models of high-level cognition postulate serial processing of task-relevant features and demonstrate that prioritizing the processing of one feature over the other can affect performance. We investigated the hypothesis that motivationally relevant task features also may be processed serially, that people may prioritize either benefit or cost information, and that artificially controlling prioritization may be beneficial for performance (by improving task-accuracy) and decision-making (by boosting the willingness to engage in effortful trials). We manipulated prioritization by altering order of presentation of effort and reward cues in two experiments involving preparation for effortful performance and effort-based decision-making. We simulated the tasks with a recent model of prefrontal cortex (Alexander & Brown in Neural Computation, 27(11), 2354-2410, 2015). Human behavior was in line with model predictions: prioritizing reward vs. effort differentially affected performance vs. decision. Prioritizing reward was beneficial for performance, showing striking increase in accuracy, especially when a large reward was offered for a difficult task. Counterintuitively (yet predicted by the model), prioritizing reward resulted in a blunted reward effect on decisions. Conversely, prioritizing effort increased reward impact on decision to engage. These results highlight the importance of controlling prioritization of motivational cues in neuroimaging studies.

Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner.

Optimal decision-making is based on integrating information from several dimensions of decisional space (e.g., reward expectation, cost estimation, effort exertion). Despite considerable empirical and theoretical efforts, the computational and neural bases of such multidimensional integration have remained largely elusive. Here we propose that the current theoretical stalemate may be broken by considering the computational properties of a cortical-subcortical circuit involving the dorsal anterior cingulate cortex (dACC) and the brainstem neuromodulatory nuclei: ventral tegmental area (VTA) and locus coeruleus (LC). From this perspective, the dACC optimizes decisions about stimuli and actions, and using the same computational machinery, it also modulates cortical functions (meta-learning), via neuromodulatory control (VTA and LC). We implemented this theory in a novel neuro-computational model-the Reinforcement Meta Learner (RML). We outline how the RML captures critical empirical findings from an unprecedented range of theoretical domains, and parsimoniously integrates various previous proposals on dACC functioning.

Prefrontal Cortex in Control: Broadening the Scope to Identify Mechanisms.

Sometime in the past two decades, neuroimaging and behavioral research converged on pFC as an important locus of cognitive control and decision-making, and that seems to be the last thing anyone has agreed on since. Every year sees an increase in the number of roles and functions attributed to distinct subregions within pFC, roles that may explain behavior and neural activity in one context but might fail to generalize across the many behaviors in which each region is implicated. Emblematic of this ongoing proliferation of functions is dorsal ACC (dACC). Novel tasks that activate dACC are followed by novel interpretations of dACC function, and each new interpretation adds to the number of functionally specific processes contained within the region. This state of affairs, a recurrent and persistent behavior followed by an illusory and transient relief, can be likened to behavioral pathology. In Journal of Cognitive Neuroscience, 29:10 we collect contributed articles that seek to move the conversation beyond specific functions of subregions of pFC, focusing instead on general roles that support pFC involvement in a wide variety of behaviors and across a variety of experimental paradigms.

Predicting Motivation: Computational Models of PFC Can Explain Neural Coding of Motivation and Effort-based Decision-making in Health and Disease.

Human behavior is strongly driven by the pursuit of rewards. In daily life, however, benefits mostly come at a cost, often requiring that effort be exerted to obtain potential benefits. Medial PFC (MPFC) and dorsolateral PFC (DLPFC) are frequently implicated in the expectation of effortful control, showing increased activity as a function of predicted task difficulty. Such activity partially overlaps with expectation of reward and has been observed both during decision-making and during task preparation. Recently, novel computational frameworks have been developed to explain activity in these regions during cognitive control, based on the principle of prediction and prediction error (predicted response-outcome [PRO] model [Alexander, W. H., & Brown, J. W. Medial prefrontal cortex as an action-outcome predictor. Nature Neuroscience, 14, 1338-1344, 2011], hierarchical error representation [HER] model [Alexander, W. H., & Brown, J. W. Hierarchical error representation: A computational model of anterior cingulate and dorsolateral prefrontal cortex. Neural Computation, 27, 2354-2410, 2015]). Despite the broad explanatory power of these models, it is not clear whether they can also accommodate effects related to the expectation of effort observed in MPFC and DLPFC. Here, we propose a translation of these computational frameworks to the domain of effort-based behavior. First, we discuss how the PRO model, based on prediction error, can explain effort-related activity in MPFC, by reframing effort-based behavior in a predictive context. We propose that MPFC activity reflects monitoring of motivationally relevant variables (such as effort and reward), by coding expectations and discrepancies from such expectations. Moreover, we derive behavioral and neural model-based predictions for healthy controls and clinical populations with impairments of motivation. Second, we illustrate the possible translation to effort-based behavior of the HER model, an extended version of PRO model based on hierarchical error prediction, developed to explain MPFC-DLPFC interactions. We derive behavioral predictions that describe how effort and reward information is coded in PFC and how changing the configuration of such environmental information might affect decision-making and task performance involving motivation.

Integrative Modeling of Prefrontal Cortex.

pFC is generally regarded as a region critical for abstract reasoning and high-level cognitive behaviors. As such, it has become the focus of intense research involving a wide variety of subdisciplines of neuroscience and employing a diverse range of methods. However, even as the amount of data on pFC has increased exponentially, it appears that progress toward understanding the general function of the region across a broad array of contexts has not kept pace. Effects observed in pFC are legion, and their interpretations are generally informed by a particular perspective or methodology with little regard with how those effects may apply more broadly. Consequently, the number of specific roles and functions that have been identified makes the region a very crowded place indeed and one that appears unlikely to be explained by a single general principle. In this theoretical article, we describe how the function of large portions of pFC can be accommodated by a single explanatory framework based on the computation and manipulation of error signals and how this framework may be extended to account for additional parts of pFC.

Unsigned value prediction-error modulates the motor system in absence of choice.

Human actions are driven by the pursuit of goals, especially when achieving these goals entails a reward. Accordingly, recent work showed that anticipating a reward in a motor task influences the motor system, boosting motor excitability and increasing overall readiness. Attaining a reward typically requires some mental or physical effort. Recent neuroimaging evidence suggested that both reward expectation and effort requirements are encoded by a partially overlapping brain network. Moreover, reward and effort information are combined in an integrative value signal. However, whether and how mental effort is integrated with reward at the motor level during task preparation remains unclear. To address these issues, we implemented a mental effort task where reward expectation and effort requirements were manipulated. During task preparation, TMS was delivered on the motor cortex and motor-evoked potentials (MEPs) were recorded on the right hand muscles to probe motor excitability. The results showed an interaction of effort and reward in modulating the motor system, reflecting an unsigned value prediction-error signal. Crucially, this was observed in the motor system in absence of a value-based decision or value-driven action selection. This suggests a high-level cognitive factor such as unsigned value prediction-error can modulate the motor system. Interestingly, effort-related motor excitability was also modulated by individual differences in tendency to engage in (and enjoy) mental effort, as measured by the Need for Cognition questionnaire, underlining a role of subjective effort experience in value-driven preparation for action.

Anticipatory processes in brain state switching - evidence from a novel cued-switching task implicating default mode and salience networks.

The default mode network (DMN) is the core brain system supporting internally oriented cognition. The ability to attenuate the DMN when switching to externally oriented processing is a prerequisite for effective performance and adaptive self-regulation. Right anterior insula (rAI), a core hub of the salience network (SN), has been proposed to control the switching from DMN to task-relevant brain networks. Little is currently known about the extent of anticipatory processes subserved by DMN and SN during switching. We investigated anticipatory DMN and SN modulation using a novel cued-switching task of between-state (rest-to-task/task-to-rest) and within-state (task-to-task) transitions. Twenty healthy adults performed the task implemented in an event-related functional magnetic resonance imaging (fMRI) design. Increases in activity were observed in the DMN regions in response to cues signalling upcoming rest. DMN attenuation was observed for rest-to-task switch cues. Obversely, DMN was up-regulated by task-to-rest cues. The strongest rAI response was observed to rest-to-task switch cues. Task-to-task switch cues elicited smaller rAI activation, whereas no significant rAI activation occurred for task-to-rest switches. Our data provide the first evidence that DMN modulation occurs rapidly and can be elicited by short duration cues signalling rest- and task-related state switches. The role of rAI appears to be limited to certain switch types - those implicating transition from a resting state and to tasks involving active cognitive engagement.

An integrative effort: Bridging motivational intensity theory and recent neurocomputational and neuronal models of effort and control allocation.

An increasing number of cognitive, neurobiological, and computational models have been proposed in the last decade, seeking to explain how humans allocate physical or cognitive effort. Most models share conceptual similarities with motivational intensity theory (MIT), an influential classic psychological theory of motivation. Yet, little effort has been made to integrate such models, which remain confined within the explanatory level for which they were developed, that is, psychological, computational, neurobiological, and neuronal. In this critical review, we derive novel analyses of three recent computational and neuronal models of effort allocation-the expected value of control theory, the reinforcement meta-learner (RML) model, and the neuronal model of attentional effort-and establish a formal relationship between these models and MIT. Our analyses reveal striking similarities between predictions made by these models, with a shared key tenet: a nonmonotonic relationship between perceived task difficulty and effort, following a sawtooth or inverted U shape. In addition, the models converge on the proposition that the dorsal anterior cingulate cortex may be responsible for determining the allocation of effort and cognitive control. We conclude by discussing the distinct contributions and strengths of each theory toward understanding neurocomputational processes of effort allocation. Finally, we highlight the necessity for a unified understanding of effort allocation, by drawing novel connections between different theorizing of adaptive effort allocation as described by the presented models. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Computational mechanisms underlying the dynamics of physical and cognitive fatigue.

The willingness to exert effort for reward is essential but comes at the cost of fatigue. Theories suggest fatigue increases after both physical and cognitive exertion, subsequently reducing the motivation to exert effort. Yet a mechanistic understanding of how this happens on a moment-to-moment basis, and whether mechanisms are common to both mental and physical effort, is lacking. In two studies, participants reported momentary (trial-by-trial) ratings of fatigue during an effort-based decision-making task requiring either physical (grip-force) or cognitive (mental arithmetic) effort. Using a novel computational model, we show that fatigue fluctuates from trial-to-trial as a function of exerted effort and predicts subsequent choices. This mechanism was shared across the domains. Selective to the cognitive domain, committing errors also induced momentary increases in feelings of fatigue. These findings provide insight into the computations underlying the influence of effortful exertion on fatigue and motivation, in both physical and cognitive domains.

Is the juice worth the squeeze? Learning the marginal value of mental effort over time.

In keeping with the view that individuals invest cognitive effort in accordance with its relative costs and benefits, reward incentives typically improve performance in tasks that require cognitive effort. At the same time, increasing effort investment may confer larger or smaller performance benefits-that is, the marginal value of effort-depending on the situation or context. On this view, we hypothesized that the magnitude of reward-induced effort modulations should depend critically on the marginal value of effort for the given context, and furthermore, the marginal value of effort of a context should be learned over time as a function of direct experience in the context. Using two well-characterized cognitive control tasks and simple computational models, we demonstrated that individuals appear to learn the marginal value of effort for different contexts. In a task-switching paradigm (Experiment 1), we found that participants initially exhibited reward-induced switch cost reductions across contexts-here, task switch rates-but over time learned to only increase effort in contexts with a comparatively larger marginal utility of effort. Similarly, in a flanker task (Experiment 2), we observed a similar learning effect across contexts defined by the proportion of incongruent trials. Together, these results enrich theories of cost-benefit effort decision-making by highlighting the importance of the (learned) marginal utility of cognitive effort. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

It's all relative: Reward-induced cognitive control modulation depends on context.

Although people seek to avoid expenditure of cognitive effort, reward incentives can increase investment of processing resources in challenging situations that require cognitive control, resulting in improved performance. At the same time, subjective value is relative, rather than absolute: The value of a reward is increased if the local context is reward-poor versus reward-rich. Although this notion is supported by work in economics and psychology, we propose that reward relativity should also play a critical role in the cost-benefit computations that inform cognitive effort allocation. Here we demonstrate that reward-induced cognitive effort allocation in a task-switching paradigm is sensitive to reward context, consistent with the notion of relative value. Informed by predictions of a computational model of divisive reward normalization, we demonstrate that reward-induced switch cost reductions depend critically upon reward context, such that the same reward amount engenders greater control allocation in impoverished versus rich reward context. Succinctly, these results confirm that reward relativity factors into the value computation driving effort allocation, revealing that motivated cognitive control, like choice, is all relative. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Opportunities and Limitations of Mobile Neuroimaging Technologies in Educational Neuroscience.

As the field of educational neuroscience continues to grow, questions have emerged regarding the ecological validity and applicability of this research to educational practice. Recent advances in mobile neuroimaging technologies have made it possible to conduct neuroscientific studies directly in naturalistic learning environments. We propose that embedding mobile neuroimaging research in a cycle (Matusz, Dikker, Huth, & Perrodin, 2019), involving lab-based, seminaturalistic, and fully naturalistic experiments, is well suited for addressing educational questions. With this review, we take a cautious approach, by discussing the valuable insights that can be gained from mobile neuroimaging technology, including electroencephalography and functional near-infrared spectroscopy, as well as the challenges posed by bringing neuroscientific methods into the classroom. Research paradigms used alongside mobile neuroimaging technology vary considerably. To illustrate this point, studies are discussed with increasingly naturalistic designs. We conclude with several ethical considerations that should be taken into account in this unique area of research.

Negative Learning Bias in Depression Revisited: Enhanced Neural Response to Surprising Reward Across Psychiatric Disorders.

BACKGROUND: Prior work has proposed that major depressive disorder (MDD) is associated with a specific cognitive bias: patients with depression seem to learn more from punishment than from reward. This learning bias has been associated with blunting of reward-related neural responses in the striatum. A key question is whether negative learning bias is also present in patients with MDD and comorbid disorders and whether this bias is specific to depression or shared across disorders. METHODS: We employed a transdiagnostic approach assessing a heterogeneous group of (nonpsychotic) psychiatric patients from the MIND-Set (Measuring Integrated Novel Dimensions in Neurodevelopmental and Stress-Related Mental Disorders) cohort with and without MDD but also with anxiety, attention-deficit/hyperactivity disorder, and/or autism (n = 66) and healthy control subjects (n = 24). To investigate reward and punishment learning, we employed a deterministic reversal learning task with functional magnetic resonance imaging. RESULTS: In contrast to previous studies, patients with MDD did not exhibit impaired reward learning or reduced reward-related neural activity anywhere in the brain. Interestingly, we observed consistently increased neural responses in the bilateral lateral prefrontal cortex of patients when they received a surprising reward. This increase was not specific to MDD, but generalized to anxiety, attention-deficit/hyperactivity disorder, and autism. Critically, increased prefrontal activity to surprising reward scaled with transdiagnostic symptom severity, particularly that associated with concentration and attention, as well as the number of diagnoses; patients with more comorbidities showed a stronger prefrontal response to surprising reward. CONCLUSIONS: Prefrontal enhancement may reflect compensatory working memory recruitment, possibly to counteract the inability to swiftly update reward expectations. This neural mechanism may provide a candidate transdiagnostic index of psychiatric severity.

Surprise, value and control in anterior cingulate cortex during speeded decision-making.

Activity in the dorsal anterior cingulate cortex (dACC) is observed across a variety of contexts, and its function remains intensely debated in the field of cognitive neuroscience. While traditional views emphasize its role in inhibitory control (suppressing prepotent, incorrect actions), recent proposals suggest a more active role in motivated control (invigorating actions to obtain rewards). Lagging behind empirical findings, formal models of dACC function primarily focus on inhibitory control, highlighting surprise, choice difficulty and value of control as key computations. Although successful in explaining dACC involvement in inhibitory control, it remains unclear whether these mechanisms generalize to motivated control. In this study, we derive predictions from three prominent accounts of dACC and test these with functional magnetic resonance imaging during value-based decision-making under time pressure. We find that the single mechanism of surprise best accounts for activity in dACC during a task requiring response invigoration, suggesting surprise signalling as a shared driver of inhibitory and motivated control.

Dissociable effects of reward magnitude on fronto-medial theta and FRN during performance monitoring.

Reward processing is influenced by reward magnitude, as previous EEG studies showed changes in amplitude of the feedback-related negativity (FRN) and reward positivity (RewP), or power of fronto-medial theta (FMθ). However, it remains unclear whether these changes are driven by increased reward sensitivity, altered reward predictions, enhanced cognitive control, or a combination of these effects. To address this question, we asked 36 participants to perform a simple gambling task where feedback valence (reward vs. no-reward), its magnitude (small vs. large reward), and expectancy (expected vs. unexpected) were manipulated in a factorial design, while 64-channel EEG was recorded concurrently. We performed standard ERP analyses (FRN and RewP) as well as time-frequency decompositions (FMθ) of feedback-locked EEG data. Subjective reports showed that large rewards were more liked and expected than small ones. At the EEG level, increasing magnitude led to a larger RewP irrespective of expectancy, whereas the FRN was not influenced by this manipulation. In comparison, FMθ power was overall increased when reward magnitude was large, except if it was unexpected. These results show dissociable effects of reward magnitude on the RewP and FMθ power. Further, they suggest, that although large reward magnitude boosts reward processing (RewP), it can nonetheless undermine the need for enhanced cognitive control (FMθ) in case reward is unexpected. We discuss these new results in terms of optimistic bias or positive mood resulting from an increased reward magnitude.