The neurocomputational link between defensive cardiac states and approach-avoidance arbitration under threat.

Avoidance, a hallmark of anxiety-related psychopathology, often comes at a cost; avoiding threat may forgo the possibility of a reward. Theories predict that optimal approach-avoidance arbitration depends on threat-induced psychophysiological states, like freezing-related bradycardia. Here we used model-based fMRI analyses to investigate whether and how bradycardia states are linked to the neurocomputational underpinnings of approach-avoidance arbitration under varying reward and threat magnitudes. We show that bradycardia states are associated with increased threat-induced avoidance and more pronounced reward-threat value comparison (i.e., a stronger tendency to approach vs. avoid when expected reward outweighs threat). An amygdala-striatal-prefrontal circuit supports approach-avoidance arbitration under threat, with specific involvement of the amygdala and dorsal anterior cingulate (dACC) in integrating reward-threat value and bradycardia states. These findings highlight the role of human freezing states in value-based decision making, relevant for optimal threat coping. They point to a specific role for amygdala/dACC in state-value integration under threat.

Goal commitment is supported by vmPFC through selective attention.

When striking a balance between commitment to a goal and flexibility in the face of better options, people often demonstrate strong goal perseveration. Here, using functional MRI (n = 30) and lesion patient (n = 26) studies, we argue that the ventromedial prefrontal cortex (vmPFC) drives goal commitment linked to changes in goal-directed selective attention. Participants performed an incremental goal pursuit task involving sequential decisions between persisting with a goal versus abandoning progress for better alternative options. Individuals with stronger goal perseveration showed higher goal-directed attention in an interleaved attention task. Increasing goal-directed attention also affected abandonment decisions: while pursuing a goal, people lost their sensitivity to valuable alternative goals while remaining more sensitive to changes in the current goal. In a healthy population, individual differences in both commitment biases and goal-oriented attention were predicted by baseline goal-related activity in the vmPFC. Among lesion patients, vmPFC damage reduced goal commitment, leading to a performance benefit.

Curiosity and the dynamics of optimal exploration.

What drives our curiosity remains an elusive and hotly debated issue, with multiple hypotheses proposed but a cohesive account yet to be established. This review discusses traditional and emergent theories that frame curiosity as a desire to know and a drive to learn, respectively. We adopt a model-based approach that maps the temporal dynamics of various factors underlying curiosity-based exploration, such as uncertainty, information gain, and learning progress. In so doing, we identify the limitations of past theories and posit an integrated account that harnesses their strengths in describing curiosity as a tool for optimal environmental exploration. In our unified account, curiosity serves as a 'common currency' for exploration, which must be balanced with other drives such as safety and hunger to achieve efficient action.

The representation of priors and decisions in the human parietal cortex.

Animals actively sample their environment through orienting actions such as saccadic eye movements. Saccadic targets are selected based both on sensory evidence immediately preceding the saccade, and a "salience map" or prior built-up over multiple saccades. In the primate cortex, the selection of each individual saccade depends on competition between target-selective cells that ramp up their firing rate to saccade release. However, it is less clear how a cross-saccade prior might be implemented, either in neural firing or through an activity-silent mechanism such as modification of synaptic weights on sensory inputs. Here, we present evidence from magnetoencephalography for 2 distinct processes underlying the selection of the current saccade, and the representation of the prior, in human parietal cortex. While the classic ramping decision process for each saccade was reflected in neural firing rates (measured in the event-related field), a prior built-up over multiple saccades was implemented via modulation of the gain on sensory inputs from the preferred target, as evidenced by rapid frequency tagging. A cascade of computations over time (initial representation of the prior, followed by evidence accumulation and then an integration of prior and evidence) provides a mechanism by which a salience map may be built up across saccades in parietal cortex. It also provides insight into the apparent contradiction that inactivation of parietal cortex has been shown not to affect performance on single-trials, despite the presence of clear evidence accumulation signals in this region.

The Computational and Neural Substrates of Ambiguity Avoidance in Anxiety.

Theoretical accounts have linked anxiety to intolerance of ambiguity. However, this relationship has not been well operationalized empirically. Here, we used computational and neuro-imaging methods to characterize anxiety-related differences in aversive decision-making under ambiguity and associated patterns of cortical activity. Adult human participants chose between two urns on each trial. The ratio of tokens ('O's and 'X's) in each urn determined probability of electrical stimulation receipt. A number above each urn indicated the magnitude of stimulation that would be received if a shock was delivered. On ambiguous trials, one of the two urns had tokens occluded. By varying the number of tokens occluded, we manipulated the extent of missing information. At higher levels of missing information, there is greater second order uncertainty, i.e., more uncertainty as to the probability of pulling a given type of token from the urn. Adult human participants demonstrated avoidance of ambiguous options which increased with level of missing information. Extent of 'information-level dependent' ambiguity aversion was significantly positively correlated with trait anxiety. Activity in both the dorsal anterior cingulate cortex and inferior frontal sulcus during the decision-making period increased as a function of missing information. Greater engagement of these regions, on high missing information trials, was observed when participants went on to select the ambiguous option; this was especially apparent in high trait anxious individuals. These findings are consistent with individuals vulnerable to anxiety requiring greater activation of frontal regions supporting rational decision-making to overcome a predisposition to engage in ambiguity avoidance at high levels of missing information.

Medial Frontal Cortex Activity Predicts Information Sampling in Economic Choice.

Decision-making not only requires agents to decide what to choose but also how much information to sample before committing to a choice. Previously established frameworks for economic choice argue for a deliberative process of evidence accumulation across time. These tacitly acknowledge a role of information sampling in that decisions are only made once sufficient evidence is acquired, yet few experiments have explicitly placed information sampling under the participant's control. Here, we use fMRI to investigate the neural basis of information sampling in economic choice by allowing participants (n = 30, sex not recorded) to actively sample information in a multistep decision task. We show that medial frontal cortex (MFC) activity is predictive of further information sampling before choice. Choice difficulty (inverse value difference, keeping sensory difficulty constant) was also encoded in MFC, but this effect was explained away by the inclusion of information sampling as a coregressor in the general linear model. A distributed network of regions across the prefrontal cortex encoded key features of the sampled information at the time it was presented. We propose that MFC is an important controller of the extent to which information is gathered before committing to an economic choice. This role may explain why MFC activity has been associated with evidence accumulation in previous studies in which information sampling was an implicit rather than explicit feature of the decision.SIGNIFICANCE STATEMENT The decisions we make are determined by the information we have sampled before committing to a choice. Accumulator frameworks of decision-making tacitly acknowledge the need to sample further information during the evidence accumulation process until a decision boundary is reached. However, relatively few studies explicitly place this decision to sample further information under the participant's control. In this fMRI study, we find that MFC activity is related to information sampling decisions in a multistep economic choice task. This suggests that an important role of evidence representations within MFC may be to guide adaptive sequential decisions to sample further information before committing to a final decision.

Defensive freezing and its relation to approach-avoidance decision-making under threat.

Successful responding to acutely threatening situations requires adequate approach-avoidance decisions. However, it is unclear how threat-induced states-like freezing-related bradycardia-impact the weighing of the potential outcomes of such value-based decisions. Insight into the underlying computations is essential, not only to improve our models of decision-making but also to improve interventions for maladaptive decisions, for instance in anxiety patients and first-responders who frequently have to make decisions under acute threat. Forty-two participants made passive and active approach-avoidance decisions under threat-of-shock when confronted with mixed outcome-prospects (i.e., varying money and shock amounts). Choice behavior was best predicted by a model including individual action-tendencies and bradycardia, beyond the subjective value of the outcome. Moreover, threat-related bradycardia (high-vs-low threat) interacted with subjective value, depending on the action-context (passive-vs-active). Specifically, in action-contexts incongruent with participants' intrinsic action-tendencies, stronger bradycardia related to diminished effects of subjective value on choice across participants. These findings illustrate the relevance of testing approach-avoidance decisions in relatively ecologically valid conditions of acute and primarily reinforced threat. These mechanistic insights into approach-avoidance conflict-resolution may inspire biofeedback-related techniques to optimize decision-making under threat. Critically, the findings demonstrate the relevance of incorporating internal psychophysiological states and external action-contexts into models of approach-avoidance decision-making.

Neuronal Computation Underlying Inferential Reasoning in Humans and Mice.

Every day we make decisions critical for adaptation and survival. We repeat actions with known consequences. But we also draw on loosely related events to infer and imagine the outcome of entirely novel choices. These inferential decisions are thought to engage a number of brain regions; however, the underlying neuronal computation remains unknown. Here, we use a multi-day cross-species approach in humans and mice to report the functional anatomy and neuronal computation underlying inferential decisions. We show that during successful inference, the mammalian brain uses a hippocampal prospective code to forecast temporally structured learned associations. Moreover, during resting behavior, coactivation of hippocampal cells in sharp-wave/ripples represent inferred relationships that include reward, thereby "joining-the-dots" between events that have not been observed together but lead to profitable outcomes. Computing mnemonic links in this manner may provide an important mechanism to build a cognitive map that stretches beyond direct experience, thus supporting flexible behavior.

Behavioral flexibility is associated with changes in structure and function distributed across a frontal cortical network in macaques.

One of the most influential accounts of central orbitofrontal cortex-that it mediates behavioral flexibility-has been challenged by the finding that discrimination reversal in macaques, the classic test of behavioral flexibility, is unaffected when lesions are made by excitotoxin injection rather than aspiration. This suggests that the critical brain circuit mediating behavioral flexibility in reversal tasks lies beyond the central orbitofrontal cortex. To determine its identity, a group of nine macaques were taught discrimination reversal learning tasks, and its impact on gray matter was measured. Magnetic resonance imaging scans were taken before and after learning and compared with scans from two control groups, each comprising 10 animals. One control group learned discrimination tasks that were similar but lacked any reversal component, and the other control group engaged in no learning. Gray matter changes were prominent in posterior orbitofrontal cortex/anterior insula but were also found in three other frontal cortical regions: lateral orbitofrontal cortex (orbital part of area 12 [12o]), cingulate cortex, and lateral prefrontal cortex. In a second analysis, neural activity in posterior orbitofrontal cortex/anterior insula was measured at rest, and its pattern of coupling with the other frontal cortical regions was assessed. Activity coupling increased significantly in the reversal learning group in comparison with controls. In a final set of experiments, we used similar structural imaging procedures and analyses to demonstrate that aspiration lesion of central orbitofrontal cortex, of the type known to affect discrimination learning, affected structure and activity in the same frontal cortical circuit. The results identify a distributed frontal cortical circuit associated with behavioral flexibility.

Control of entropy in neural models of environmental state.

Humans and animals construct internal models of their environment in order to select appropriate courses of action. The representation of uncertainty about the current state of the environment is a key feature of these models that controls the rate of learning as well as directly affecting choice behaviour. To maintain flexibility, given that uncertainty naturally decreases over time, most theoretical inference models include a dedicated mechanism to drive up model uncertainty. Here we probe the long-standing hypothesis that noradrenaline is involved in determining the uncertainty, or entropy, and thus flexibility, of neural models. Pupil diameter, which indexes neuromodulatory state including noradrenaline release, predicted increases (but not decreases) in entropy in a neural state model encoded in human medial orbitofrontal cortex, as measured using multivariate functional MRI. Activity in anterior cingulate cortex predicted pupil diameter. These results provide evidence for top-down, neuromodulatory control of entropy in neural state models.

A Network for Computing Value Equilibrium in the Human Medial Prefrontal Cortex.

Humans and other animals make decisions in order to satisfy their goals. However, it remains unknown how neural circuits compute which of multiple possible goals should be pursued (e.g., when balancing hunger and thirst) and how to combine these signals with estimates of available reward alternatives. Here, humans undergoing fMRI accumulated two distinct assets over a sequence of trials. Financial outcomes depended on the minimum cumulate of either asset, creating a need to maintain "value equilibrium" by redressing any imbalance among the assets. Blood-oxygen-level-dependent (BOLD) signals in the rostral anterior cingulate cortex (rACC) tracked the level of imbalance among goals, whereas the ventromedial prefrontal cortex (vmPFC) signaled the level of redress incurred by a choice rather than the overall amount received. These results suggest that a network of medial frontal brain regions compute a value signal that maintains value equilibrium among internal goals.

State-change decisions and dorsomedial prefrontal cortex: the importance of time.

Different kinds of decision making can be categorized by their differential effect on the agent's current and future states as well as the computational challenges they pose. Here, we draw a distinction between within-state and state-change decision-making, and propose that a dedicated decision mechanism exists in dorsomedial prefrontal cortex (dmPFC) that is specialized for state-change decisions. We set out a formal framework in which state change decisions may be made on the basis of the integrated momentary reward rate, over the intended time to be spent in a state. A key feature of this framework is that reward rate is expressed as a function of continuous time. We argue that dmPFC is suited for this type of decision making partly due to its ability to track the passage of time. This proposed function of dmPFC is placed in contrast to other evaluative systems such as the orbitofrontal cortex, which is important for careful deliberation within a specific model-space or option-space and within a decision strategy.

Towards a neuro-computational account of prism adaptation.

Prism adaptation has a long history as an experimental paradigm used to investigate the functional and neural processes that underlie sensorimotor control. In the neuropsychology literature, prism adaptation behaviour is typically explained by reference to a traditional cognitive psychology framework that distinguishes putative functions, such as 'strategic control' versus 'spatial realignment'. This theoretical framework lacks conceptual clarity, quantitative precision and explanatory power. Here, we advocate for an alternative computational framework that offers several advantages: 1) an algorithmic explanatory account of the computations and operations that drive behaviour; 2) expressed in quantitative mathematical terms; 3) embedded within a principled theoretical framework (Bayesian decision theory, state-space modelling); 4) that offers a means to generate and test quantitative behavioural predictions. This computational framework offers a route towards mechanistic neurocognitive explanations of prism adaptation behaviour. Thus it constitutes a conceptual advance compared to the traditional theoretical framework. In this paper, we illustrate how Bayesian decision theory and state-space models offer principled explanations for a range of behavioural phenomena in the field of prism adaptation (e.g. visual capture, magnitude of visual versus proprioceptive realignment, spontaneous recovery and dynamics of adaptation memory). We argue that this explanatory framework can advance understanding of the functional and neural mechanisms that implement prism adaptation behaviour, by enabling quantitative tests of hypotheses that go beyond merely descriptive mapping claims that 'brain area X is (somehow) involved in psychological process Y'.

Organizing conceptual knowledge in humans with a gridlike code.

It has been hypothesized that the brain organizes concepts into a mental map, allowing conceptual relationships to be navigated in a manner similar to that of space. Grid cells use a hexagonally symmetric code to organize spatial representations and are the likely source of a precise hexagonal symmetry in the functional magnetic resonance imaging signal. Humans navigating conceptual two-dimensional knowledge showed the same hexagonal signal in a set of brain regions markedly similar to those activated during spatial navigation. This gridlike signal is consistent across sessions acquired within an hour and more than a week apart. Our findings suggest that global relational codes may be used to organize nonspatial conceptual representations and that these codes may have a hexagonal gridlike pattern when conceptual knowledge is laid out in two continuous dimensions.

Two Anatomically and Computationally Distinct Learning Signals Predict Changes to Stimulus-Outcome Associations in Hippocampus.

Complex cognitive processes require sophisticated local processing but also interactions between distant brain regions. It is therefore critical to be able to study distant interactions between local computations and the neural representations they act on. Here we report two anatomically and computationally distinct learning signals in lateral orbitofrontal cortex (lOFC) and the dopaminergic ventral midbrain (VM) that predict trial-by-trial changes to a basic internal model in hippocampus. To measure local computations during learning and their interaction with neural representations, we coupled computational fMRI with trial-by-trial fMRI suppression. We find that suppression in a medial temporal lobe network changes trial-by-trial in proportion to stimulus-outcome associations. During interleaved choice trials, we identify learning signals that relate to outcome type in lOFC and to reward value in VM. These intervening choice feedback signals predicted the subsequent change to hippocampal suppression, suggesting a convergence of signals that update the flexible representation of stimulus-outcome associations.

Testing the inter-hemispheric competition account of visual extinction with combined TMS/fMRI.

Theoretical models of visual neglect and extinction entail claims about the normal functioning of attention and parietal cortex in the healthy brain: (1) 'pseudoneglect', a commonly observed attentional bias towards left space, reflects the greater dominance of parietal cortex activity of the right versus left hemisphere; (2) the capacity to distribute attention bilaterally depends causally on the relative balance of parietal activity between the hemispheres; (3) disruption of the dominant right parietal cortex shifts this inter-hemispheric balance leftward, causing a rightward shift in attentional bias. We tested these claims using low-frequency offline transcranial magnetic stimulation (TMS) to transiently inhibit activity in the right angular gyrus/intra-parietal sulcus, followed by a visual detection task to assess changes in attentional bias, and functional magnetic resonance imaging (fMRI) to test for the predicted leftward shift in brain activity. The task required participants to covertly monitor both hemifields to detect and report the location of upcoming transient visual targets that appeared on the left, right or bilaterally. In the behavioural experiment, participants exhibited a leftward attentional bias ('pseudoneglect') at baseline, which was abolished by TMS. In the fMRI experiment, participants activated an expected network of visual, parietal and frontal cortex bilaterally during the period of covert bilateral attention. TMS shifted the relative hemispheric balance of parietal activity from right to left. The consistent direction of TMS-induced behavioural and functional change indicates a causal role for parietal inter-hemispheric balance in distributing visual attention across space.

Anxious individuals have difficulty learning the causal statistics of aversive environments.

Statistical regularities in the causal structure of the environment enable us to predict the probable outcomes of our actions. Environments differ in the extent to which action-outcome contingencies are stable or volatile. Difficulty in being able to use this information to optimally update outcome predictions might contribute to the decision-making difficulties seen in anxiety. We tested this using an aversive learning task manipulating environmental volatility. Human participants low in trait anxiety matched updating of their outcome predictions to the volatility of the current environment, as predicted by a Bayesian model. Individuals with high trait anxiety showed less ability to adjust updating of outcome expectancies between stable and volatile environments. This was linked to reduced sensitivity of the pupil dilatory response to volatility, potentially indicative of altered norepinephrinergic responsivity to changes in this aspect of environmental information.

Causal manipulation of functional connectivity in a specific neural pathway during behaviour and at rest.

Correlations in brain activity between two areas (functional connectivity) have been shown to relate to their underlying structural connections. We examine the possibility that functional connectivity also reflects short-term changes in synaptic efficacy. We demonstrate that paired transcranial magnetic stimulation (TMS) near ventral premotor cortex (PMv) and primary motor cortex (M1) with a short 8-ms inter-pulse interval evoking synchronous pre- and post-synaptic activity and which strengthens interregional connectivity between the two areas in a pattern consistent with Hebbian plasticity, leads to increased functional connectivity between PMv and M1 as measured with functional magnetic resonance imaging (fMRI). Moreover, we show that strengthening connectivity between these nodes has effects on a wider network of areas, such as decreasing coupling in a parallel motor programming stream. A control experiment revealed that identical TMS pulses at identical frequencies caused no change in fMRI-measured functional connectivity when the inter-pulse-interval was too long for Hebbian-like plasticity.

A neural circuit covarying with social hierarchy in macaques.

Despite widespread interest in social dominance, little is known of its neural correlates in primates. We hypothesized that social status in primates might be related to individual variation in subcortical brain regions implicated in other aspects of social and emotional behavior in other mammals. To examine this possibility we used magnetic resonance imaging (MRI), which affords the taking of quantitative measurements noninvasively, both of brain structure and of brain function, across many regions simultaneously. We carried out a series of tests of structural and functional MRI (fMRI) data in 25 group-living macaques. First, a deformation-based morphometric (DBM) approach was used to show that gray matter in the amygdala, brainstem in the vicinity of the raphe nucleus, and reticular formation, hypothalamus, and septum/striatum of the left hemisphere was correlated with social status. Second, similar correlations were found in the same areas in the other hemisphere. Third, similar correlations were found in a second data set acquired several months later from a subset of the same animals. Fourth, the strength of coupling between fMRI-measured activity in the same areas was correlated with social status. The network of subcortical areas, however, had no relationship with the sizes of individuals' social networks, suggesting the areas had a simple and direct relationship with social status. By contrast a second circuit in cortex, comprising the midsuperior temporal sulcus and anterior and dorsal prefrontal cortex, covaried with both individuals' social statuses and the social network sizes they experienced. This cortical circuit may be linked to the social cognitive processes that are taxed by life in more complex social networks and that must also be used if an animal is to achieve a high social status.

Brain systems for probabilistic and dynamic prediction: computational specificity and integration.

A computational approach to functional specialization suggests that brain systems can be characterized in terms of the types of computations they perform, rather than their sensory or behavioral domains. We contrasted the neural systems associated with two computationally distinct forms of predictive model: a reinforcement-learning model of the environment obtained through experience with discrete events, and continuous dynamic forward modeling. By manipulating the precision with which each type of prediction could be used, we caused participants to shift computational strategies within a single spatial prediction task. Hence (using fMRI) we showed that activity in two brain systems (typically associated with reward learning and motor control) could be dissociated in terms of the forms of computations that were performed there, even when both systems were used to make parallel predictions of the same event. A region in parietal cortex, which was sensitive to the divergence between the predictions of the models and anatomically connected to both computational networks, is proposed to mediate integration of the two predictive modes to produce a single behavioral output.