Dissociation and integration of outcome and state uncertainty signals in cognitive control.

Signals related to uncertainty are frequently observed in regions of the cognitive control network, including anterior cingulate/medial prefrontal cortex (ACC/mPFC), dorsolateral prefrontal cortex (dlPFC), and anterior insular cortex. Uncertainty generally refers to conditions in which decision variables may assume multiple possible values and can arise at multiple points in the perception-action cycle, including sensory input, inferred states of the environment, and the consequences of actions. These sources of uncertainty are frequently correlated: noisy input can lead to unreliable estimates of the state of the environment, with consequential influences on action selection. Given this correlation amongst various sources of uncertainty, dissociating the neural structures underlying their estimation presents an ongoing issue: a region associated with uncertainty related to outcomes may estimate outcome uncertainty itself, or it may reflect a cascade effect of state uncertainty on outcome estimates. In this study, we derive signals of state and outcome uncertainty from mathematical models of risk and observe regions in the cognitive control network whose activity is best explained by signals related to state uncertainty (anterior insula), outcome uncertainty (dlPFC), as well as regions that appear to integrate the two (ACC/mPFC).

Degeneracy measures in biologically plausible random Boolean networks.

BACKGROUND: Degeneracy-the ability of structurally different elements to perform similar functions-is a property of many biological systems. Highly degenerate systems show resilience to perturbations and damage because the system can compensate for compromised function due to reconfiguration of the underlying network dynamics. Degeneracy thus suggests how biological systems can thrive despite changes to internal and external demands. Although degeneracy is a feature of network topologies and seems to be implicated in a wide variety of biological processes, research on degeneracy in biological networks is mostly limited to weighted networks. In this study, we test an information theoretic definition of degeneracy on random Boolean networks, frequently used to model gene regulatory networks. Random Boolean networks are discrete dynamical systems with binary connectivity and thus, these networks are well-suited for tracing information flow and the causal effects. By generating networks with random binary wiring diagrams, we test the effects of systematic lesioning of connections and perturbations of the network nodes on the degeneracy measure. RESULTS: Our analysis shows that degeneracy, on average, is the highest in networks in which ~ 20% of the connections are lesioned while 50% of the nodes are perturbed. Moreover, our results for the networks with no lesions and the fully-lesioned networks are comparable to the degeneracy measures from weighted networks, thus we show that the degeneracy measure is applicable to different networks. CONCLUSIONS: Such a generalized applicability implies that degeneracy measures may be a useful tool for investigating a wide range of biological networks and, therefore, can be used to make predictions about the variety of systems' ability to recover function.

Task-specific prioritization of reward and effort information: Novel insights from behavior and computational modeling.

Efficient integration of environmental information is critical in goal-directed behavior. Motivational information regarding potential rewards and costs (such as required effort) affects performance and decisions whether to engage in a task. While it is generally acknowledged that costs and benefits are integrated to determine the level of effort to be exerted, how this integration occurs remains an open question. Computational models of high-level cognition postulate serial processing of task-relevant features and demonstrate that prioritizing the processing of one feature over the other can affect performance. We investigated the hypothesis that motivationally relevant task features also may be processed serially, that people may prioritize either benefit or cost information, and that artificially controlling prioritization may be beneficial for performance (by improving task-accuracy) and decision-making (by boosting the willingness to engage in effortful trials). We manipulated prioritization by altering order of presentation of effort and reward cues in two experiments involving preparation for effortful performance and effort-based decision-making. We simulated the tasks with a recent model of prefrontal cortex (Alexander & Brown in Neural Computation, 27(11), 2354-2410, 2015). Human behavior was in line with model predictions: prioritizing reward vs. effort differentially affected performance vs. decision. Prioritizing reward was beneficial for performance, showing striking increase in accuracy, especially when a large reward was offered for a difficult task. Counterintuitively (yet predicted by the model), prioritizing reward resulted in a blunted reward effect on decisions. Conversely, prioritizing effort increased reward impact on decision to engage. These results highlight the importance of controlling prioritization of motivational cues in neuroimaging studies.

Prefrontal Cortex in Control: Broadening the Scope to Identify Mechanisms.

Sometime in the past two decades, neuroimaging and behavioral research converged on pFC as an important locus of cognitive control and decision-making, and that seems to be the last thing anyone has agreed on since. Every year sees an increase in the number of roles and functions attributed to distinct subregions within pFC, roles that may explain behavior and neural activity in one context but might fail to generalize across the many behaviors in which each region is implicated. Emblematic of this ongoing proliferation of functions is dorsal ACC (dACC). Novel tasks that activate dACC are followed by novel interpretations of dACC function, and each new interpretation adds to the number of functionally specific processes contained within the region. This state of affairs, a recurrent and persistent behavior followed by an illusory and transient relief, can be likened to behavioral pathology. In Journal of Cognitive Neuroscience, 29:10 we collect contributed articles that seek to move the conversation beyond specific functions of subregions of pFC, focusing instead on general roles that support pFC involvement in a wide variety of behaviors and across a variety of experimental paradigms.

Distinct Regions within Medial Prefrontal Cortex Process Pain and Cognition.

Neuroimaging studies of the medial prefrontal cortex (mPFC) suggest that the dorsal anterior cingulate cortex (dACC) region is responsive to a wide variety of stimuli and psychological states, such as pain, cognitive control, and prediction error (PE). In contrast, a recent meta-analysis argues that the dACC is selective for pain, whereas the supplementary motor area (SMA) and pre-SMA are specifically associated with higher-level cognitive processes (Lieberman and Eisenberger, 2015). To empirically test this claim, we manipulated effects of pain, conflict, and PE in a single experiment using human subjects. We observed a robust dorsal-ventral dissociation within the mPFC with cognitive effects of PE and conflict overlapping dorsally and pain localized more ventrally. Classification of subjects based on the presence or absence of a paracingulate sulcus showed that PE effects extended across the dorsal area of the dACC and into the pre-SMA. These results begin to resolve recent controversies by showing the following: (1) the mPFC includes dissociable regions for pain and cognitive processing; and (2) meta-analyses are correct in localizing cognitive effects to the dACC, although these effects extend to the pre-SMA as well. These results both provide evidence distinguishing between different theories of mPFC function and highlight the importance of taking individual anatomical variability into account when conducting empirical studies of the mPFC. SIGNIFICANCE STATEMENT: Decades of neuroimaging research have shown the mPFC to represent a wide variety of stimulus processing and cognitive states. However, recently it has been argued whether distinct regions of the mPFC separately process pain and cognitive phenomena. To address this controversy, this study directly compared pain and cognitive processes within subjects. We found a double dissociation within the mPFC with pain localized ventral to the cingulate sulcus and cognitive effects localized more dorsally within the dACC and spreading into the pre-supplementary motor area. This provides empirical evidence to help resolve the current debate about the functional architecture of the mPFC.

Predicting Motivation: Computational Models of PFC Can Explain Neural Coding of Motivation and Effort-based Decision-making in Health and Disease.

Human behavior is strongly driven by the pursuit of rewards. In daily life, however, benefits mostly come at a cost, often requiring that effort be exerted to obtain potential benefits. Medial PFC (MPFC) and dorsolateral PFC (DLPFC) are frequently implicated in the expectation of effortful control, showing increased activity as a function of predicted task difficulty. Such activity partially overlaps with expectation of reward and has been observed both during decision-making and during task preparation. Recently, novel computational frameworks have been developed to explain activity in these regions during cognitive control, based on the principle of prediction and prediction error (predicted response-outcome [PRO] model [Alexander, W. H., & Brown, J. W. Medial prefrontal cortex as an action-outcome predictor. Nature Neuroscience, 14, 1338-1344, 2011], hierarchical error representation [HER] model [Alexander, W. H., & Brown, J. W. Hierarchical error representation: A computational model of anterior cingulate and dorsolateral prefrontal cortex. Neural Computation, 27, 2354-2410, 2015]). Despite the broad explanatory power of these models, it is not clear whether they can also accommodate effects related to the expectation of effort observed in MPFC and DLPFC. Here, we propose a translation of these computational frameworks to the domain of effort-based behavior. First, we discuss how the PRO model, based on prediction error, can explain effort-related activity in MPFC, by reframing effort-based behavior in a predictive context. We propose that MPFC activity reflects monitoring of motivationally relevant variables (such as effort and reward), by coding expectations and discrepancies from such expectations. Moreover, we derive behavioral and neural model-based predictions for healthy controls and clinical populations with impairments of motivation. Second, we illustrate the possible translation to effort-based behavior of the HER model, an extended version of PRO model based on hierarchical error prediction, developed to explain MPFC-DLPFC interactions. We derive behavioral predictions that describe how effort and reward information is coded in PFC and how changing the configuration of such environmental information might affect decision-making and task performance involving motivation.

Foraging Value, Risk Avoidance, and Multiple Control Signals: How the Anterior Cingulate Cortex Controls Value-based Decision-making.

Recent work on the role of the ACC in cognition has focused on choice difficulty, action value, risk avoidance, conflict resolution, and the value of exerting control among other factors. A main underlying question is what are the output signals of ACC, and relatedly, what is their effect on downstream cognitive processes? Here we propose a model of how ACC influences cognitive processing in other brain regions that choose actions. The model builds on the earlier Predicted Response Outcome model and suggests that ACC learns to represent specifically the states in which the potential costs or risks of an action are high, on both short and long timescales. It then uses those cost signals as a basis to bias decisions to minimize losses while maximizing gains. The model simulates both proactive and reactive control signals and accounts for a variety of empirical findings regarding value-based decision-making.

Integrative Modeling of Prefrontal Cortex.

pFC is generally regarded as a region critical for abstract reasoning and high-level cognitive behaviors. As such, it has become the focus of intense research involving a wide variety of subdisciplines of neuroscience and employing a diverse range of methods. However, even as the amount of data on pFC has increased exponentially, it appears that progress toward understanding the general function of the region across a broad array of contexts has not kept pace. Effects observed in pFC are legion, and their interpretations are generally informed by a particular perspective or methodology with little regard with how those effects may apply more broadly. Consequently, the number of specific roles and functions that have been identified makes the region a very crowded place indeed and one that appears unlikely to be explained by a single general principle. In this theoretical article, we describe how the function of large portions of pFC can be accommodated by a single explanatory framework based on the computation and manipulation of error signals and how this framework may be extended to account for additional parts of pFC.

Hierarchical Error Representation: A Computational Model of Anterior Cingulate and Dorsolateral Prefrontal Cortex.

Anterior cingulate and dorsolateral prefrontal cortex (ACC and dlPFC, respectively) are core components of the cognitive control network. Activation of these regions is routinely observed in tasks that involve monitoring the external environment and maintaining information in order to generate appropriate responses. Despite the ubiquity of studies reporting coactivation of these two regions, a consensus on how they interact to support cognitive control has yet to emerge. In this letter, we present a new hypothesis and computational model of ACC and dlPFC. The error representation hypothesis states that multidimensional error signals generated by ACC in response to surprising outcomes are used to train representations of expected error in dlPFC, which are then associated with relevant task stimuli. Error representations maintained in dlPFC are in turn used to modulate predictive activity in ACC in order to generate better estimates of the likely outcomes of actions. We formalize the error representation hypothesis in a new computational model based on our previous model of ACC. The hierarchical error representation (HER) model of ACC/dlPFC suggests a mechanism by which hierarchically organized layers within ACC and dlPFC interact in order to solve sophisticated cognitive tasks. In a series of simulations, we demonstrate the ability of the HER model to autonomously learn to perform structured tasks in a manner comparable to human performance, and we show that the HER model outperforms current deep learning networks by an order of magnitude.

Distinct regions of anterior cingulate cortex signal prediction and outcome evaluation.

A number of theories have been proposed to account for the role of anterior cingulate cortex (ACC) and the broader medial prefrontal cortex (mPFC) in cognition. The recent Prediction of Response Outcome (PRO) computational model casts the mPFC in part as performing two theoretically distinct functions: learning to predict the various possible outcomes of actions, and then evaluating those predictions against the actual outcomes. Simulations have shown that this new model can account for an unprecedented range of known mPFC effects, but the central theory of distinct prediction and evaluation mechanisms within ACC remains untested. Using combined computational neural modeling and fMRI, we show here that prediction and evaluation signals are indeed each represented in the ACC, and furthermore, they are represented in distinct regions within ACC. Our task independently manipulated both the number of predicted outcomes and the degree to which outcomes violated expectancies, the former providing assessment of regions sensitive to prediction and the latter providing assessment of regions sensitive to evaluation. Using quantitative regressors derived from the PRO computational model, we show that prediction-based model signals load on a network including the posterior and perigenual ACC, but outcome evaluation model signals load on the mid-dorsal ACC. These findings are consistent with distinct prediction and evaluation signals as posited by the PRO model and provide new perspective on a large set of known effects within ACC.

Independent and interacting value systems for reward and information in the human brain.

Theories of prefrontal cortex (PFC) as optimizing reward value have been widely deployed to explain its activity in a diverse range of contexts, with substantial empirical support in neuroeconomics and decision neuroscience. Similar neural circuits, however, have also been associated with information processing. By using computational modeling, model-based functional magnetic resonance imaging analysis, and a novel experimental paradigm, we aim at establishing whether a dedicated and independent value system for information exists in the human PFC. We identify two regions in the human PFC that independently encode reward and information. Our results provide empirical evidence for PFC as an optimizer of independent information and reward signals during decision-making under realistic scenarios, with potential implications for the interpretation of PFC activity in both healthy and clinical populations.

Multisensory integration in humans with spinal cord injury.

Although multisensory integration (MSI) has been extensively studied, the underlying mechanisms remain a topic of ongoing debate. Here we investigate these mechanisms by comparing MSI in healthy controls to a clinical population with spinal cord injury (SCI). Deafferentation following SCI induces sensorimotor impairment, which may alter the ability to synthesize cross-modal information. We applied mathematical and computational modeling to reaction time data recorded in response to temporally congruent cross-modal stimuli. We found that MSI in both SCI and healthy controls is best explained by cross-modal perceptual competition, highlighting a common competition mechanism. Relative to controls, MSI impairments in SCI participants were better explained by reduced stimulus salience leading to increased cross-modal competition. By combining traditional analyses with model-based approaches, we examine how MSI is realized during normal function, and how it is compromised in a clinical population. Our findings support future investigations identifying and rehabilitating MSI deficits in clinical disorders.

Interactions of Medial and Lateral Prefrontal Cortex in Hierarchical Predictive Coding.

Cognitive control and decision-making rely on the interplay of medial and lateral prefrontal cortex (mPFC/lPFC), particularly for circumstances in which correct behavior requires integrating and selecting among multiple sources of interrelated information. While the interaction between mPFC and lPFC is generally acknowledged as a crucial circuit in adaptive behavior, the nature of this interaction remains open to debate, with various proposals suggesting complementary roles in (i) signaling the need for and implementing control, (ii) identifying and selecting appropriate behavioral policies from a candidate set, and (iii) constructing behavioral schemata for performance of structured tasks. Although these proposed roles capture salient aspects of conjoint mPFC/lPFC function, none are sufficiently well-specified to provide a detailed account of the continuous interaction of the two regions during ongoing behavior. A recent computational model of mPFC and lPFC, the Hierarchical Error Representation (HER) model, places the regions within the framework of hierarchical predictive coding, and suggests how they interact during behavioral periods preceding and following salient events. In this manuscript, we extend the HER model to incorporate real-time temporal dynamics and demonstrate how the extended model is able to capture single-unit neurophysiological, behavioral, and network effects previously reported in the literature. Our results add to the wide range of results that can be accounted for by the HER model, and provide further evidence for predictive coding as a unifying framework for understanding PFC function and organization.

Competition between learned reward and error outcome predictions in anterior cingulate cortex.

The anterior cingulate cortex (ACC) is implicated in performance monitoring and cognitive control. Non-human primate studies of ACC show prominent reward signals, but these are elusive in human studies, which instead show mainly conflict and error effects. Here we demonstrate distinct appetitive and aversive activity in human ACC. The error likelihood hypothesis suggests that ACC activity increases in proportion to the likelihood of an error, and ACC is also sensitive to the consequence magnitude of the predicted error. Previous work further showed that error likelihood effects reach a ceiling as the potential consequences of an error increase, possibly due to reductions in the average reward. We explored this issue by independently manipulating reward magnitude of task responses and error likelihood while controlling for potential error consequences in an Incentive Change Signal Task. The fMRI results ruled out a modulatory effect of expected reward on error likelihood effects in favor of a competition effect between expected reward and error likelihood. Dynamic causal modeling showed that error likelihood and expected reward signals are intrinsic to the ACC rather than received from elsewhere. These findings agree with interpretations of ACC activity as signaling both perceptions of risk and predicted reward.

Hyperbolically discounted temporal difference learning.

Hyperbolic discounting of future outcomes is widely observed to underlie choice behavior in animals. Additionally, recent studies (Kobayashi & Schultz, 2008) have reported that hyperbolic discounting is observed even in neural systems underlying choice. However, the most prevalent models of temporal discounting, such as temporal difference learning, assume that future outcomes are discounted exponentially. Exponential discounting has been preferred largely because it can be expressed recursively, whereas hyperbolic discounting has heretofore been thought not to have a recursive definition. In this letter, we define a learning algorithm, hyperbolically discounted temporal difference (HDTD) learning, which constitutes a recursive formulation of the hyperbolic model.

Are we there yet? The four-year impact of a VA fellowship program on the recovery orientation of rehabilitation programs.

OBJECTIVE: This study represents the first program evaluation of the impact of a Psychosocial Rehabilitation (PSR) fellowship program within the Veterans Health Administration (VHA). Specifically, it examines the recovery orientation of five mental health rehabilitation programs at the Edith Nourse Rogers Memorial VA Medical Center (ENRM VAMC) in Bedford, MA by comparing program stakeholder rating of the "recovery orientation" between the initial data and the four-year follow-up during which the PSR fellowship was in operation. The goal of this fellowship program is to increase the VHA's fidelity to recovery-oriented best practice recommendations. METHOD: Participants were mental health consumers and staff members within five key psychiatric rehabilitation programs at the ENRM VAMC. Perception of programs' recovery orientation was measured at the start of the fellowship (Time 1) and after the fellowship was in place for four years (Time 2). RESULTS: Results demonstrate that across the entire sample of stakeholders, perceptions of recovery orientation significantly improved from Time 1 to Time 2. Results also reveal a significant overall increase in program recovery orientation over time in three out of the five rehabilitation programs, with years of fellow involvement in particular programs significantly and positively correlating with increases in ratings of program recovery-orientation gains. DISCUSSION: Implications for using fellowships as agents of program change, and specifically, recovery-oriented change, are discussed.

Surprise, value and control in anterior cingulate cortex during speeded decision-making.

Activity in the dorsal anterior cingulate cortex (dACC) is observed across a variety of contexts, and its function remains intensely debated in the field of cognitive neuroscience. While traditional views emphasize its role in inhibitory control (suppressing prepotent, incorrect actions), recent proposals suggest a more active role in motivated control (invigorating actions to obtain rewards). Lagging behind empirical findings, formal models of dACC function primarily focus on inhibitory control, highlighting surprise, choice difficulty and value of control as key computations. Although successful in explaining dACC involvement in inhibitory control, it remains unclear whether these mechanisms generalize to motivated control. In this study, we derive predictions from three prominent accounts of dACC and test these with functional magnetic resonance imaging during value-based decision-making under time pressure. We find that the single mechanism of surprise best accounts for activity in dACC during a task requiring response invigoration, suggesting surprise signalling as a shared driver of inhibitory and motivated control.

The Role of the Anterior Cingulate Cortex in Prediction Error and Signaling Surprise.

In the past two decades, reinforcement learning (RL) has become a popular framework for understanding brain function. A key component of RL models, prediction error, has been associated with neural signals throughout the brain, including subcortical nuclei, primary sensory cortices, and prefrontal cortex. Depending on the location in which activity is observed, the functional interpretation of prediction error may change: Prediction errors may reflect a discrepancy in the anticipated and actual value of reward, a signal indicating the salience or novelty of a stimulus, and many other interpretations. Anterior cingulate cortex (ACC) has long been recognized as a region involved in processing behavioral error, and recent computational models of the region have expanded this interpretation to include a more general role for the region in predicting likely events, broadly construed, and signaling deviations between expected and observed events. Ongoing modeling work investigating the interaction between ACC and additional regions involved in cognitive control suggests an even broader role for cingulate in computing a hierarchically structured surprise signal critical for learning models of the environment. The result is a predictive coding model of the frontal lobes, suggesting that predictive coding may be a unifying computational principle across the neocortex.

Frontal cortex function as derived from hierarchical predictive coding.

The frontal lobes are essential for human volition and goal-directed behavior, yet their function remains unclear. While various models have highlighted working memory, reinforcement learning, and cognitive control as key functions, a single framework for interpreting the range of effects observed in prefrontal cortex has yet to emerge. Here we show that a simple computational motif based on predictive coding can be stacked hierarchically to learn and perform arbitrarily complex goal-directed behavior. The resulting Hierarchical Error Representation (HER) model simulates a wide array of findings from fMRI, ERP, single-units, and neuropsychological studies of both lateral and medial prefrontal cortex. By reconceptualizing lateral prefrontal activity as anticipating prediction errors, the HER model provides a novel unifying account of prefrontal cortex function with broad implications for understanding the frontal cortex across multiple levels of description, from the level of single neurons to behavior.

Fast, Accurate, and Stable Feature Selection Using Neural Networks.

Multi-voxel pattern analysis often necessitates feature selection due to the high dimensional nature of neuroimaging data. In this context, feature selection techniques serve the dual purpose of potentially increasing classification accuracy and revealing sets of features that best discriminate between classes. However, feature selection techniques in current, widespread use in the literature suffer from a number of deficits, including the need for extended computational time, lack of consistency in selecting features relevant to classification, and only marginal increases in classifier accuracy. In this paper we present a novel method for feature selection based on a single-layer neural network which incorporates cross-validation during feature selection and stability selection through iterative subsampling. Comparing our approach to popular alternative feature selection methods, we find increased classifier accuracy, reduced computational cost and greater consistency with which relevant features are selected. Furthermore, we demonstrate that importance mapping, a technique used to identify voxels relevant to classification, can lead to the selection of irrelevant voxels due to shared activation patterns across categories. Our method, owing to its relatively simple architecture, flexibility and speed, can provide a viable alternative for researchers to identify sets of features that best discriminate classes.