Neuronal Computation Underlying Inferential Reasoning in Humans and Mice.

Every day we make decisions critical for adaptation and survival. We repeat actions with known consequences. But we also draw on loosely related events to infer and imagine the outcome of entirely novel choices. These inferential decisions are thought to engage a number of brain regions; however, the underlying neuronal computation remains unknown. Here, we use a multi-day cross-species approach in humans and mice to report the functional anatomy and neuronal computation underlying inferential decisions. We show that during successful inference, the mammalian brain uses a hippocampal prospective code to forecast temporally structured learned associations. Moreover, during resting behavior, coactivation of hippocampal cells in sharp-wave/ripples represent inferred relationships that include reward, thereby "joining-the-dots" between events that have not been observed together but lead to profitable outcomes. Computing mnemonic links in this manner may provide an important mechanism to build a cognitive map that stretches beyond direct experience, thus supporting flexible behavior.

The representation of priors and decisions in the human parietal cortex.

Animals actively sample their environment through orienting actions such as saccadic eye movements. Saccadic targets are selected based both on sensory evidence immediately preceding the saccade, and a "salience map" or prior built-up over multiple saccades. In the primate cortex, the selection of each individual saccade depends on competition between target-selective cells that ramp up their firing rate to saccade release. However, it is less clear how a cross-saccade prior might be implemented, either in neural firing or through an activity-silent mechanism such as modification of synaptic weights on sensory inputs. Here, we present evidence from magnetoencephalography for 2 distinct processes underlying the selection of the current saccade, and the representation of the prior, in human parietal cortex. While the classic ramping decision process for each saccade was reflected in neural firing rates (measured in the event-related field), a prior built-up over multiple saccades was implemented via modulation of the gain on sensory inputs from the preferred target, as evidenced by rapid frequency tagging. A cascade of computations over time (initial representation of the prior, followed by evidence accumulation and then an integration of prior and evidence) provides a mechanism by which a salience map may be built up across saccades in parietal cortex. It also provides insight into the apparent contradiction that inactivation of parietal cortex has been shown not to affect performance on single-trials, despite the presence of clear evidence accumulation signals in this region.

Behavioral flexibility is associated with changes in structure and function distributed across a frontal cortical network in macaques.

One of the most influential accounts of central orbitofrontal cortex-that it mediates behavioral flexibility-has been challenged by the finding that discrimination reversal in macaques, the classic test of behavioral flexibility, is unaffected when lesions are made by excitotoxin injection rather than aspiration. This suggests that the critical brain circuit mediating behavioral flexibility in reversal tasks lies beyond the central orbitofrontal cortex. To determine its identity, a group of nine macaques were taught discrimination reversal learning tasks, and its impact on gray matter was measured. Magnetic resonance imaging scans were taken before and after learning and compared with scans from two control groups, each comprising 10 animals. One control group learned discrimination tasks that were similar but lacked any reversal component, and the other control group engaged in no learning. Gray matter changes were prominent in posterior orbitofrontal cortex/anterior insula but were also found in three other frontal cortical regions: lateral orbitofrontal cortex (orbital part of area 12 [12o]), cingulate cortex, and lateral prefrontal cortex. In a second analysis, neural activity in posterior orbitofrontal cortex/anterior insula was measured at rest, and its pattern of coupling with the other frontal cortical regions was assessed. Activity coupling increased significantly in the reversal learning group in comparison with controls. In a final set of experiments, we used similar structural imaging procedures and analyses to demonstrate that aspiration lesion of central orbitofrontal cortex, of the type known to affect discrimination learning, affected structure and activity in the same frontal cortical circuit. The results identify a distributed frontal cortical circuit associated with behavioral flexibility.

Medial Frontal Cortex Activity Predicts Information Sampling in Economic Choice.

Decision-making not only requires agents to decide what to choose but also how much information to sample before committing to a choice. Previously established frameworks for economic choice argue for a deliberative process of evidence accumulation across time. These tacitly acknowledge a role of information sampling in that decisions are only made once sufficient evidence is acquired, yet few experiments have explicitly placed information sampling under the participant's control. Here, we use fMRI to investigate the neural basis of information sampling in economic choice by allowing participants (n = 30, sex not recorded) to actively sample information in a multistep decision task. We show that medial frontal cortex (MFC) activity is predictive of further information sampling before choice. Choice difficulty (inverse value difference, keeping sensory difficulty constant) was also encoded in MFC, but this effect was explained away by the inclusion of information sampling as a coregressor in the general linear model. A distributed network of regions across the prefrontal cortex encoded key features of the sampled information at the time it was presented. We propose that MFC is an important controller of the extent to which information is gathered before committing to an economic choice. This role may explain why MFC activity has been associated with evidence accumulation in previous studies in which information sampling was an implicit rather than explicit feature of the decision.SIGNIFICANCE STATEMENT The decisions we make are determined by the information we have sampled before committing to a choice. Accumulator frameworks of decision-making tacitly acknowledge the need to sample further information during the evidence accumulation process until a decision boundary is reached. However, relatively few studies explicitly place this decision to sample further information under the participant's control. In this fMRI study, we find that MFC activity is related to information sampling decisions in a multistep economic choice task. This suggests that an important role of evidence representations within MFC may be to guide adaptive sequential decisions to sample further information before committing to a final decision.

A neural circuit covarying with social hierarchy in macaques.

Despite widespread interest in social dominance, little is known of its neural correlates in primates. We hypothesized that social status in primates might be related to individual variation in subcortical brain regions implicated in other aspects of social and emotional behavior in other mammals. To examine this possibility we used magnetic resonance imaging (MRI), which affords the taking of quantitative measurements noninvasively, both of brain structure and of brain function, across many regions simultaneously. We carried out a series of tests of structural and functional MRI (fMRI) data in 25 group-living macaques. First, a deformation-based morphometric (DBM) approach was used to show that gray matter in the amygdala, brainstem in the vicinity of the raphe nucleus, and reticular formation, hypothalamus, and septum/striatum of the left hemisphere was correlated with social status. Second, similar correlations were found in the same areas in the other hemisphere. Third, similar correlations were found in a second data set acquired several months later from a subset of the same animals. Fourth, the strength of coupling between fMRI-measured activity in the same areas was correlated with social status. The network of subcortical areas, however, had no relationship with the sizes of individuals' social networks, suggesting the areas had a simple and direct relationship with social status. By contrast a second circuit in cortex, comprising the midsuperior temporal sulcus and anterior and dorsal prefrontal cortex, covaried with both individuals' social statuses and the social network sizes they experienced. This cortical circuit may be linked to the social cognitive processes that are taxed by life in more complex social networks and that must also be used if an animal is to achieve a high social status.

Brain systems for probabilistic and dynamic prediction: computational specificity and integration.

A computational approach to functional specialization suggests that brain systems can be characterized in terms of the types of computations they perform, rather than their sensory or behavioral domains. We contrasted the neural systems associated with two computationally distinct forms of predictive model: a reinforcement-learning model of the environment obtained through experience with discrete events, and continuous dynamic forward modeling. By manipulating the precision with which each type of prediction could be used, we caused participants to shift computational strategies within a single spatial prediction task. Hence (using fMRI) we showed that activity in two brain systems (typically associated with reward learning and motor control) could be dissociated in terms of the forms of computations that were performed there, even when both systems were used to make parallel predictions of the same event. A region in parietal cortex, which was sensitive to the divergence between the predictions of the models and anatomically connected to both computational networks, is proposed to mediate integration of the two predictive modes to produce a single behavioral output.

Dissociable effects of surprise and model update in parietal and anterior cingulate cortex.

Brains use predictive models to facilitate the processing of expected stimuli or planned actions. Under a predictive model, surprising (low probability) stimuli or actions necessitate the immediate reallocation of processing resources, but they can also signal the need to update the underlying predictive model to reflect changes in the environment. Surprise and updating are often correlated in experimental paradigms but are, in fact, distinct constructs that can be formally defined as the Shannon information (IS) and Kullback-Leibler divergence (DKL) associated with an observation. In a saccadic planning task, we observed that distinct behaviors and brain regions are associated with surprise/IS and updating/DKL. Although surprise/IS was associated with behavioral reprogramming as indexed by slower reaction times, as well as with activity in the posterior parietal cortex [human lateral intraparietal area (LIP)], the anterior cingulate cortex (ACC) was specifically activated during updating of the predictive model (DKL). A second saccade-sensitive region in the inferior posterior parietal cortex (human 7a), which has connections to both LIP and ACC, was activated by surprise and modulated by updating. Pupillometry revealed a further dissociation between surprise and updating with an early positive effect of surprise and late negative effect of updating on pupil area. These results give a computational account of the roles of the ACC and two parietal saccade regions, LIP and 7a, by which their involvement in diverse tasks can be understood mechanistically. The dissociation of functional roles between regions within the reorienting/reprogramming network may also inform models of neurological phenomena, such as extinction and Balint syndrome, and neglect.

Causal effect of disconnection lesions on interhemispheric functional connectivity in rhesus monkeys.

In the absence of external stimuli or task demands, correlations in spontaneous brain activity (functional connectivity) reflect patterns of anatomical connectivity. Hence, resting-state functional connectivity has been used as a proxy measure for structural connectivity and as a biomarker for brain changes in disease. To relate changes in functional connectivity to physiological changes in the brain, it is important to understand how correlations in functional connectivity depend on the physical integrity of brain tissue. The causal nature of this relationship has been called into question by patient data suggesting that decreased structural connectivity does not necessarily lead to decreased functional connectivity. Here we provide evidence for a causal but complex relationship between structural connectivity and functional connectivity: we tested interhemispheric functional connectivity before and after corpus callosum section in rhesus monkeys. We found that forebrain commissurotomy severely reduced interhemispheric functional connectivity, but surprisingly, this effect was greatly mitigated if the anterior commissure was left intact. Furthermore, intact structural connections increased their functional connectivity in line with the hypothesis that the inputs to each node are normalized. We conclude that functional connectivity is likely driven by corticocortical white matter connections but with complex network interactions such that a near-normal pattern of functional connectivity can be maintained by just a few indirect structural connections. These surprising results highlight the importance of network-level interactions in functional connectivity and may cast light on various paradoxical findings concerning changes in functional connectivity in disease states.

The organization of dorsal frontal cortex in humans and macaques.

The human dorsal frontal cortex has been associated with the most sophisticated aspects of cognition, including those that are thought to be especially refined in humans. Here we used diffusion-weighted magnetic resonance imaging (DW-MRI) and functional MRI (fMRI) in humans and macaques to infer and compare the organization of dorsal frontal cortex in the two species. Using DW-MRI tractography-based parcellation, we identified 10 dorsal frontal regions lying between the human inferior frontal sulcus and cingulate cortex. Patterns of functional coupling between each area and the rest of the brain were then estimated with fMRI and compared with functional coupling patterns in macaques. Areas in human medial frontal cortex, including areas associated with high-level social cognitive processes such as theory of mind, showed a surprising degree of similarity in their functional coupling patterns with the frontal pole, medial prefrontal, and dorsal prefrontal convexity in the macaque. We failed to find evidence for "new" regions in human medial frontal cortex. On the lateral surface, comparison of functional coupling patterns suggested correspondences in anatomical organization distinct from those that are widely assumed. A human region sometimes referred to as lateral frontal pole more closely resembled area 46, rather than the frontal pole, of the macaque. Overall the pattern of results suggest important similarities in frontal cortex organization in humans and other primates, even in the case of regions thought to carry out uniquely human functions. The patterns of interspecies correspondences are not, however, always those that are widely assumed.

Curiosity and the dynamics of optimal exploration.

What drives our curiosity remains an elusive and hotly debated issue, with multiple hypotheses proposed but a cohesive account yet to be established. This review discusses traditional and emergent theories that frame curiosity as a desire to know and a drive to learn, respectively. We adopt a model-based approach that maps the temporal dynamics of various factors underlying curiosity-based exploration, such as uncertainty, information gain, and learning progress. In so doing, we identify the limitations of past theories and posit an integrated account that harnesses their strengths in describing curiosity as a tool for optimal environmental exploration. In our unified account, curiosity serves as a 'common currency' for exploration, which must be balanced with other drives such as safety and hunger to achieve efficient action.

Goal commitment is supported by vmPFC through selective attention.

When striking a balance between commitment to a goal and flexibility in the face of better options, people often demonstrate strong goal perseveration. Here, using functional MRI (n = 30) and lesion patient (n = 26) studies, we argue that the ventromedial prefrontal cortex (vmPFC) drives goal commitment linked to changes in goal-directed selective attention. Participants performed an incremental goal pursuit task involving sequential decisions between persisting with a goal versus abandoning progress for better alternative options. Individuals with stronger goal perseveration showed higher goal-directed attention in an interleaved attention task. Increasing goal-directed attention also affected abandonment decisions: while pursuing a goal, people lost their sensitivity to valuable alternative goals while remaining more sensitive to changes in the current goal. In a healthy population, individual differences in both commitment biases and goal-oriented attention were predicted by baseline goal-related activity in the vmPFC. Among lesion patients, vmPFC damage reduced goal commitment, leading to a performance benefit.

The neurocomputational link between defensive cardiac states and approach-avoidance arbitration under threat.

Avoidance, a hallmark of anxiety-related psychopathology, often comes at a cost; avoiding threat may forgo the possibility of a reward. Theories predict that optimal approach-avoidance arbitration depends on threat-induced psychophysiological states, like freezing-related bradycardia. Here we used model-based fMRI analyses to investigate whether and how bradycardia states are linked to the neurocomputational underpinnings of approach-avoidance arbitration under varying reward and threat magnitudes. We show that bradycardia states are associated with increased threat-induced avoidance and more pronounced reward-threat value comparison (i.e., a stronger tendency to approach vs. avoid when expected reward outweighs threat). An amygdala-striatal-prefrontal circuit supports approach-avoidance arbitration under threat, with specific involvement of the amygdala and dorsal anterior cingulate (dACC) in integrating reward-threat value and bradycardia states. These findings highlight the role of human freezing states in value-based decision making, relevant for optimal threat coping. They point to a specific role for amygdala/dACC in state-value integration under threat.

Diffusion-weighted imaging tractography-based parcellation of the human parietal cortex and comparison with human and macaque resting-state functional connectivity.

Despite the prominence of parietal activity in human neuroimaging investigations of sensorimotor and cognitive processes, there remains uncertainty about basic aspects of parietal cortical anatomical organization. Descriptions of human parietal cortex draw heavily on anatomical schemes developed in other primate species, but the validity of such comparisons has been questioned by claims that there are fundamental differences between the parietal cortex in humans and other primates. A scheme is presented for parcellation of human lateral parietal cortex into component regions on the basis of anatomical connectivity and the functional interactions of the resulting clusters with other brain regions. Anatomical connectivity was estimated using diffusion-weighted magnetic resonance image (MRI)-based tractography, and functional interactions were assessed by correlations in activity measured with functional MRI at rest. Resting-state functional connectivity was also assessed directly in the rhesus macaque lateral parietal cortex in an additional experiment, and the patterns found reflected known neuroanatomical connections. Cross-correlation in the tractography-based connectivity patterns of parietal voxels reliably parcellated human lateral parietal cortex into 10 component clusters. The resting-state functional connectivity of human superior parietal and intraparietal clusters with frontal and extrastriate cortex suggested correspondences with areas in macaque superior and intraparietal sulcus. Functional connectivity patterns with parahippocampal cortex and premotor cortex again suggested fundamental correspondences between inferior parietal cortex in humans and macaques. In contrast, the human parietal cortex differs in the strength of its interactions between the central inferior parietal lobule region and the anterior prefrontal cortex.

How can a Bayesian approach inform neuroscience?

In this review we consider how Bayesian logic can help neuroscientists to understand behaviour and brain function. Firstly, we review some key characteristics of Bayesian systems - they integrate information making rational use of uncertainty, they apply prior knowledge in the interpretation of new observations, and (for several reasons) they are very effective learners. Secondly, we illustrate how some well-known psychological phenomena including visual illusions, categorical perception and attention can be understood in terms of Bayesian inference. We also consider how formal models can clarify our understanding of psychological constructs, by giving a truly computational definition of psychological processes. Finally, we consider how probabilistic representations and hence Bayesian algorithms could be implemented by neural populations. In particular, we explore how different types of population coding may lead to different predictions about activity in both single-unit and imaging studies, and draw a distinction in this context between the representation of parameters and implementation of computations.

The Computational and Neural Substrates of Ambiguity Avoidance in Anxiety.

Theoretical accounts have linked anxiety to intolerance of ambiguity. However, this relationship has not been well operationalized empirically. Here, we used computational and neuro-imaging methods to characterize anxiety-related differences in aversive decision-making under ambiguity and associated patterns of cortical activity. Adult human participants chose between two urns on each trial. The ratio of tokens ('O's and 'X's) in each urn determined probability of electrical stimulation receipt. A number above each urn indicated the magnitude of stimulation that would be received if a shock was delivered. On ambiguous trials, one of the two urns had tokens occluded. By varying the number of tokens occluded, we manipulated the extent of missing information. At higher levels of missing information, there is greater second order uncertainty, i.e., more uncertainty as to the probability of pulling a given type of token from the urn. Adult human participants demonstrated avoidance of ambiguous options which increased with level of missing information. Extent of 'information-level dependent' ambiguity aversion was significantly positively correlated with trait anxiety. Activity in both the dorsal anterior cingulate cortex and inferior frontal sulcus during the decision-making period increased as a function of missing information. Greater engagement of these regions, on high missing information trials, was observed when participants went on to select the ambiguous option; this was especially apparent in high trait anxious individuals. These findings are consistent with individuals vulnerable to anxiety requiring greater activation of frontal regions supporting rational decision-making to overcome a predisposition to engage in ambiguity avoidance at high levels of missing information.

Distinct and overlapping functional zones in the cerebellum defined by resting state functional connectivity.

The cerebellum processes information from functionally diverse regions of the cerebral cortex. Cerebellar input and output nuclei have connections with prefrontal, parietal, and sensory cortex as well as motor and premotor cortex. However, the topography of the connections between the cerebellar and cerebral cortices remains largely unmapped, as it is relatively unamenable to anatomical methods. We used resting-state functional magnetic resonance imaging to define subregions within the cerebellar cortex based on their functional connectivity with the cerebral cortex. We mapped resting-state functional connectivity voxel-wise across the cerebellar cortex, for cerebral-cortical masks covering prefrontal, motor, somatosensory, posterior parietal, visual, and auditory cortices. We found that the cerebellum can be divided into at least 2 zones: 1) a primary sensorimotor zone (Lobules V, VI, and VIII), which contains overlapping functional connectivity maps for domain-specific motor, somatosensory, visual, and auditory cortices; and 2) a supramodal zone (Lobules VIIa, Crus I, and II), which contains overlapping functional connectivity maps for prefrontal and posterior-parietal cortex. The cortical connectivity of the supramodal zone was driven by regions of frontal and parietal cortex which are not directly involved in sensory or motor processing, including dorsolateral prefrontal cortex and the frontal pole, and the inferior parietal lobule.

Defensive freezing and its relation to approach-avoidance decision-making under threat.

Successful responding to acutely threatening situations requires adequate approach-avoidance decisions. However, it is unclear how threat-induced states-like freezing-related bradycardia-impact the weighing of the potential outcomes of such value-based decisions. Insight into the underlying computations is essential, not only to improve our models of decision-making but also to improve interventions for maladaptive decisions, for instance in anxiety patients and first-responders who frequently have to make decisions under acute threat. Forty-two participants made passive and active approach-avoidance decisions under threat-of-shock when confronted with mixed outcome-prospects (i.e., varying money and shock amounts). Choice behavior was best predicted by a model including individual action-tendencies and bradycardia, beyond the subjective value of the outcome. Moreover, threat-related bradycardia (high-vs-low threat) interacted with subjective value, depending on the action-context (passive-vs-active). Specifically, in action-contexts incongruent with participants' intrinsic action-tendencies, stronger bradycardia related to diminished effects of subjective value on choice across participants. These findings illustrate the relevance of testing approach-avoidance decisions in relatively ecologically valid conditions of acute and primarily reinforced threat. These mechanistic insights into approach-avoidance conflict-resolution may inspire biofeedback-related techniques to optimize decision-making under threat. Critically, the findings demonstrate the relevance of incorporating internal psychophysiological states and external action-contexts into models of approach-avoidance decision-making.

Effort-based cost-benefit valuation and the human brain.

In both the wild and the laboratory, animals' preferences for one course of action over another reflect not just reward expectations but also the cost in terms of effort that must be invested in pursuing the course of action. The ventral striatum and dorsal anterior cingulate cortex (ACCd) are implicated in the making of cost-benefit decisions in the rat, but there is little information about how effort costs are processed and influence calculations of expected net value in other mammals including humans. We performed a functional magnetic resonance imaging study to determine whether and where activity in the human brain was available to guide effort-based cost-benefit valuation. Subjects were scanned while they performed a series of effortful actions to obtain secondary reinforcers. At the beginning of each trial, subjects were presented with one of eight different visual cues that they had learned indicated how much effort the course of action would entail and how much reward could be expected at its completion. Cue-locked activity in the ventral striatum and midbrain reflected the net value of the course of action, signaling the expected amount of reward discounted by the amount of effort to be invested. Activity in ACCd also reflected the interaction of both expected reward and effort costs. Posterior orbitofrontal and insular activity, however, only reflected the expected reward magnitude. The ventral striatum and anterior cingulate cortex may be the substrate of effort-based cost-benefit valuation in primates as well as in rats.

The cerebellum predicts the timing of perceptual events.

Prospective (forward) temporal-spatial models are essential for both action and perception, but the literature on perceptual prediction has primarily been limited to the spatial domain. In this study we asked how the neural systems of perceptual prediction change, when change-over-time must be modeled. We used a naturalistic paradigm in which observers had to extrapolate the trajectory of an occluded moving object to make perceptual judgments based on the spatial (direction) or temporal-spatial (velocity) characteristics of object motion. Using functional magnetic resonance imaging we found that a region in posterior cerebellum (lobule VII crus 1) was engaged specifically when a temporal-spatial model was required (velocity judgment task), suggesting that circuitry involved in motor forward-modeling may also be engaged in perceptual prediction when a model of change-over-time is required. This cerebellar region appears to supply a temporal signal to cortical networks involved in spatial orienting: a frontal-parietal network associated with attentional orienting was engaged in both (spatial and temporal-spatial) tasks, but functional connectivity between these regions and the posterior cerebellum was enhanced in the temporal-spatial prediction task. In addition to the oculomotor spatial orienting network, regions involved in hand movements (aIP and PMv) were recruited in the temporal-spatial task, suggesting that the nature of perceptual prediction may bias the recruitment of sensory-motor networks in orienting. Finally, in temporal-spatial prediction, functional connectivity was enhanced between the cerebellum and the putamen, a structure which has been proposed to supply the brain's metric of time, in the temporal-spatial prediction task.

Control of entropy in neural models of environmental state.

Humans and animals construct internal models of their environment in order to select appropriate courses of action. The representation of uncertainty about the current state of the environment is a key feature of these models that controls the rate of learning as well as directly affecting choice behaviour. To maintain flexibility, given that uncertainty naturally decreases over time, most theoretical inference models include a dedicated mechanism to drive up model uncertainty. Here we probe the long-standing hypothesis that noradrenaline is involved in determining the uncertainty, or entropy, and thus flexibility, of neural models. Pupil diameter, which indexes neuromodulatory state including noradrenaline release, predicted increases (but not decreases) in entropy in a neural state model encoded in human medial orbitofrontal cortex, as measured using multivariate functional MRI. Activity in anterior cingulate cortex predicted pupil diameter. These results provide evidence for top-down, neuromodulatory control of entropy in neural state models.