What's Worth the Risk? A Neural Circuit for Trade-Offs.

Cost-benefit analysis in decision making takes place in everyday life for animals and humans alike. In this issue, a neural circuit specific for modulating these behaviors is identified in rats and reveals elusive functional distinctions between long-mysterious anatomical features of the brain.

A Common Space Approach to Comparative Neuroscience.

Comparative neuroscience is entering the era of big data. New high-throughput methods and data-sharing initiatives have resulted in the availability of large, digital data sets containing many types of data from ever more species. Here, we present a framework for exploiting the new possibilities offered. The multimodality of the data allows vertical translations, which are comparisons of different aspects of brain organization within a single species and across scales. Horizontal translations compare particular aspects of brain organization across species, often by building abstract feature spaces. Combining vertical and horizontal translations allows for more sophisticated comparisons, including relating principles of brain organization across species by contrasting horizontal translations, and for making formal predictions of unobtainable data based on observed results in a model species.

Activation and disruption of a neural mechanism for novel choice in monkeys.

Neural mechanisms that mediate the ability to make value-guided decisions have received substantial attention in humans and animals1-6. Experiments in animals typically involve long training periods. By contrast, choices in the real world often need to be made between new options spontaneously. It is therefore possible that the neural mechanisms targeted in animal studies differ from those required for new decisions, which are typical of human imaging studies. Here we show that the primate medial frontal cortex (MFC)7 is involved in making new inferential choices when the options have not been previously experienced. Macaques spontaneously inferred the values of new options via similarities with the component parts of previously encountered options. Functional magnetic resonance imaging (fMRI) suggested that this ability was mediated by the MFC, which is rarely investigated in monkeys3; MFC activity reflected different processes of comparison for unfamiliar and familiar options. Multidimensional representations of options in the MFC used a coding scheme resembling that of grid cells, which is well known in spatial navigation8,9, to integrate dimensions in this non-physical space10 during novel decision-making. By contrast, the orbitofrontal cortex held specific object-based value representations1,11. In addition, minimally invasive ultrasonic disruption12 of MFC, but not adjacent tissue, altered the estimation of novel choice values.

Neural Mechanisms of Social Cognition in Primates.

Activity in a network of areas spanning the superior temporal sulcus, dorsomedial frontal cortex, and anterior cingulate cortex is concerned with how nonhuman primates negotiate the social worlds in which they live. Central aspects of these circuits are retained in humans. Activity in these areas codes for primates' interactions with one another, their attempts to find out about one another, and their attempts to prevent others from finding out too much about themselves. Moreover, important features of the social world, such as dominance status, cooperation, and competition, modulate activity in these areas. We consider the degree to which activity in these regions is simply encoding an individual's own actions and choices or whether this activity is especially and specifically concerned with social cognition. Recent advances in comparative anatomy and computational modeling may help us to gain deeper insights into the nature and boundaries of primate social cognition.

The effect of apathy and compulsivity on planning and stopping in sequential decision-making.

Real-life decision-making often comprises sequences of successive decisions about whether to take opportunities as they are encountered or keep searching for better ones instead. We investigated individual differences related to such sequential decision-making and link them especially to apathy and compulsivity in a large online sample (discovery sample: n = 449 and confirmation sample: n = 756). Our cognitive model revealed distinct changes in the way participants evaluated their environments and planned their own future behaviour. Apathy was linked to decision inertia, i.e., automatically persisting with a sequence of searches for longer than appropriate given the value of searching. Thus, despite being less motivated, they did not avoid the effort associated with longer searches. In contrast, compulsivity was linked to self-reported insensitivity to the cost of continuing with a sequence of searches. The objective measures of behavioural cost insensitivity were clearly linked to compulsivity only in the discovery sample. While the confirmation sample showed a similar effect, it did not reach significance. Nevertheless, in both samples, participants reported awareness of such bias (experienced as "overchasing"). In addition, this awareness made them report preemptively avoiding situations related to the bias. However, we found no evidence of them actually preempting more in the task, which might mean a misalignment of their metacognitive beliefs or that our behavioural measures were incomplete. In summary, individual variation in distinct, fundamental aspects of sequential decision-making can be linked to variation in 2 measures of behavioural traits associated with psychological illness in the normal population.

Increasing and decreasing interregional brain coupling increases and decreases oscillatory activity in the human brain.

The origins of oscillatory activity in the brain are currently debated, but common to many hypotheses is the notion that they reflect interactions between brain areas. Here, we examine this possibility by manipulating the strength of coupling between two human brain regions, ventral premotor cortex (PMv) and primary motor cortex (M1), and examine the impact on oscillatory activity in the motor system measurable in the electroencephalogram. We either increased or decreased the strength of coupling while holding the impact on each component area in the pathway constant. This was achieved by stimulating PMv and M1 with paired pulses of transcranial magnetic stimulation using two different patterns, only one of which increases the influence exerted by PMv over M1. While the stimulation protocols differed in their temporal patterning, they were comprised of identical numbers of pulses to M1 and PMv. We measured the impact on activity in alpha, beta, and theta bands during a motor task in which participants either made a preprepared action (Go) or withheld it (No-Go). Augmenting cortical connectivity between PMv and M1, by evoking synchronous pre- and postsynaptic activity in the PMv-M1 pathway, enhanced oscillatory beta and theta rhythms in Go and No-Go trials, respectively. Little change was observed in the alpha rhythm. By contrast, diminishing the influence of PMv over M1 decreased oscillatory beta and theta rhythms in Go and No-Go trials, respectively. This suggests that corticocortical communication frequencies in the PMv-M1 pathway can be manipulated following Hebbian spike-timing-dependent plasticity.

Multiple systems in macaques for tracking prediction errors and other types of surprise.

Animals learn from the past to make predictions. These predictions are adjusted after prediction errors, i.e., after surprising events. Generally, most reward prediction errors models learn the average expected amount of reward. However, here we demonstrate the existence of distinct mechanisms for detecting other types of surprising events. Six macaques learned to respond to visual stimuli to receive varying amounts of juice rewards. Most trials ended with the delivery of either 1 or 3 juice drops so that animals learned to expect 2 juice drops on average even though instances of precisely 2 drops were rare. To encourage learning, we also included sessions during which the ratio between 1 and 3 drops changed. Additionally, in all sessions, the stimulus sometimes appeared in an unexpected location. Thus, 3 types of surprising events could occur: reward amount surprise (i.e., a scalar reward prediction error), rare reward surprise, and visuospatial surprise. Importantly, we can dissociate scalar reward prediction errors-rewards that deviated from the average reward amount expected-and rare reward events-rewards that accorded with the average reward expectation but that rarely occurred. We linked each type of surprise to a distinct pattern of neural activity using functional magnetic resonance imaging. Activity in the vicinity of the dopaminergic midbrain only reflected surprise about the amount of reward. Lateral prefrontal cortex had a more general role in detecting surprising events. Posterior lateral orbitofrontal cortex specifically detected rare reward events regardless of whether they followed average reward amount expectations, but only in learnable reward environments.

Behavioral flexibility is associated with changes in structure and function distributed across a frontal cortical network in macaques.

One of the most influential accounts of central orbitofrontal cortex-that it mediates behavioral flexibility-has been challenged by the finding that discrimination reversal in macaques, the classic test of behavioral flexibility, is unaffected when lesions are made by excitotoxin injection rather than aspiration. This suggests that the critical brain circuit mediating behavioral flexibility in reversal tasks lies beyond the central orbitofrontal cortex. To determine its identity, a group of nine macaques were taught discrimination reversal learning tasks, and its impact on gray matter was measured. Magnetic resonance imaging scans were taken before and after learning and compared with scans from two control groups, each comprising 10 animals. One control group learned discrimination tasks that were similar but lacked any reversal component, and the other control group engaged in no learning. Gray matter changes were prominent in posterior orbitofrontal cortex/anterior insula but were also found in three other frontal cortical regions: lateral orbitofrontal cortex (orbital part of area 12 [12o]), cingulate cortex, and lateral prefrontal cortex. In a second analysis, neural activity in posterior orbitofrontal cortex/anterior insula was measured at rest, and its pattern of coupling with the other frontal cortical regions was assessed. Activity coupling increased significantly in the reversal learning group in comparison with controls. In a final set of experiments, we used similar structural imaging procedures and analyses to demonstrate that aspiration lesion of central orbitofrontal cortex, of the type known to affect discrimination learning, affected structure and activity in the same frontal cortical circuit. The results identify a distributed frontal cortical circuit associated with behavioral flexibility.

Human decisions about when to act originate within a basal forebrain-nigral circuit.

Decisions about when to act are critical for survival in humans as in animals, but how a desire is translated into the decision that an action is worth taking at any particular point in time is incompletely understood. Here we show that a simple model developed to explain when animals decide it is worth taking an action also explains a significant portion of the variance in timing observed when humans take voluntary actions. The model focuses on the current environment's potential for reward, the timing of the individual's own recent actions, and the outcomes of those actions. We show, by using ultrahigh-field MRI scanning, that in addition to anterior cingulate cortex within medial frontal cortex, a group of subcortical structures including striatum, substantia nigra, basal forebrain (BF), pedunculopontine nucleus (PPN), and habenula (HB) encode trial-by-trial variation in action time. Further analysis of the activity patterns found in each area together with psychophysiological interaction analysis and structural equation modeling suggested a model in which BF integrates contextual information that will influence the decision about when to act and communicates this information, in parallel with PPN and HB influences, to nigrostriatal circuits. It is then in the nigrostriatal circuit that action initiation per se begins.

Differential functional connectivity underlying asymmetric reward-related activity in human and nonhuman primates.

The orbitofrontal cortex (OFC) is a key brain region involved in complex cognitive functions such as reward processing and decision making. Neuroimaging studies have reported unilateral OFC response to reward-related variables; however, those studies rarely discussed this observation. Nevertheless, some lesion studies suggest that the left and right OFC contribute differently to cognitive processes. We hypothesized that the OFC asymmetrical response to reward could reflect underlying hemispherical difference in OFC functional connectivity. Using resting-state and reward-related functional MRI data from humans and from rhesus macaques, we first identified an asymmetrical response of the lateral OFC to reward in both species. Crucially, the subregion showing the highest reward-related asymmetry (RRA) overlapped with the region showing the highest functional connectivity asymmetry (FCA). Furthermore, the two types of asymmetries were found to be significantly correlated across individuals. In both species, the right lateral OFC was more connected to the default mode network compared to the left lateral OFC. Altogether, our results suggest a functional specialization of the left and right lateral OFC in primates.

Previously Reward-Associated Stimuli Capture Spatial Attention in the Absence of Changes in the Corresponding Sensory Representations as Measured with MEG.

Studies of selective attention typically consider the role of task goals or physical salience, but attention can also be captured by previously reward-associated stimuli, even if they are currently task irrelevant. One theory underlying this value-driven attentional capture (VDAC) is that reward-associated stimulus representations undergo plasticity in sensory cortex, thereby automatically capturing attention during early processing. To test this, we used magnetoencephalography to probe whether stimulus location and identity representations in sensory cortex are modulated by reward learning. We furthermore investigated the time course of these neural effects, and their relationship to behavioral VDAC. Male and female human participants first learned stimulus-reward associations. Next, we measured VDAC in a separate task by presenting these stimuli in the absence of reward contingency and probing their effects on the processing of separate target stimuli presented at different time lags. Using time-resolved multivariate pattern analysis, we found that learned value modulated the spatial selection of previously rewarded stimuli in posterior visual and parietal cortex from ∼260 ms after stimulus onset. This value modulation was related to the strength of participants' behavioral VDAC effect and persisted into subsequent target processing. Importantly, learned value did not influence cortical signatures of early processing (i.e., earlier than ∼200 ms); nor did it influence the decodability of stimulus identity. Our results suggest that VDAC is underpinned by learned value signals that modulate spatial selection throughout posterior visual and parietal cortex. We further suggest that VDAC can occur in the absence of changes in early visual processing in cortex.SIGNIFICANCE STATEMENT Attention is our ability to focus on relevant information at the expense of irrelevant information. It can be affected by previously learned but currently irrelevant stimulus-reward associations, a phenomenon termed "value-driven attentional capture" (VDAC). The neural mechanisms underlying VDAC remain unclear. It has been speculated that reward learning induces visual cortical plasticity, which modulates early visual processing to capture attention. Although we find that learned value modulates spatial signals in visual cortical areas, an effect that correlates with VDAC, we find no relevant signatures of changes in early visual processing in cortex.

Combining brain perturbation and neuroimaging in non-human primates.

Brain perturbation studies allow detailed causal inferences of behavioral and neural processes. Because the combination of brain perturbation methods and neural measurement techniques is inherently challenging, research in humans has predominantly focused on non-invasive, indirect brain perturbations, or neurological lesion studies. Non-human primates have been indispensable as a neurobiological system that is highly similar to humans while simultaneously being more experimentally tractable, allowing visualization of the functional and structural impact of systematic brain perturbation. This review considers the state of the art in non-human primate brain perturbation with a focus on approaches that can be combined with neuroimaging. We consider both non-reversible (lesions) and reversible or temporary perturbations such as electrical, pharmacological, optical, optogenetic, chemogenetic, pathway-selective, and ultrasound based interference methods. Method-specific considerations from the research and development community are offered to facilitate research in this field and support further innovations. We conclude by identifying novel avenues for further research and innovation and by highlighting the clinical translational potential of the methods.

Beyond negative valence: 2-week administration of a serotonergic antidepressant enhances both reward and effort learning signals.

To make good decisions, humans need to learn about and integrate different sources of appetitive and aversive information. While serotonin has been linked to value-based decision-making, its role in learning is less clear, with acute manipulations often producing inconsistent results. Here, we show that when the effects of a selective serotonin reuptake inhibitor (SSRI, citalopram) are studied over longer timescales, learning is robustly improved. We measured brain activity with functional magnetic resonance imaging (fMRI) in volunteers as they performed a concurrent appetitive (money) and aversive (effort) learning task. We found that 2 weeks of citalopram enhanced reward and effort learning signals in a widespread network of brain regions, including ventromedial prefrontal and anterior cingulate cortex. At a behavioral level, this was accompanied by more robust reward learning. This suggests that serotonin can modulate the ability to learn via a mechanism that is independent of stimulus valence. Such effects may partly underlie SSRIs' impact in treating psychological illnesses. Our results highlight both a specific function in learning for serotonin and the importance of studying its role across longer timescales.

Contrasting Effects of Medial and Lateral Orbitofrontal Cortex Lesions on Credit Assignment and Decision-Making in Humans.

The orbitofrontal cortex is critical for goal-directed behavior. Recent work in macaques has suggested the lateral orbitofrontal cortex (lOFC) is relatively more concerned with assignment of credit for rewards to particular choices during value-guided learning, whereas the medial orbitofrontal cortex (often referred to as ventromedial prefrontal cortex in humans; vmPFC/mOFC) is involved in constraining the decision to the relevant options. We examined whether people with damage restricted to subregions of prefrontal cortex showed the patterns of impairment observed in prior investigations of the effects of lesions to homologous regions in macaques. Groups of patients with either lOFC (predominantly right hemisphere), mOFC/vmPFC, or dorsomedial prefrontal (DMF), and a comparison group of healthy age- and education-matched controls performed a probabilistic 3-choice decision-making task. We report anatomically specific patterns of impairment. We found that credit assignment, as indexed by the normal influence of contingent relationships between choice and reward, is reduced in lOFC patients compared with Controls and mOFC/vmPFC patients. Moreover, the effects of reward contingency on choice were similar for patients with lesions in DMF or mOFC/vmPFC, compared with Controls. By contrast, mOFC/vmPFC-lesioned patients made more stochastic choices than Controls when the decision was framed by valuable distracting alternatives, suggesting that value comparisons were no longer independent of irrelevant options. Once again, there was evidence of regional specialization: patients with lOFC lesions were unimpaired relative to Controls. As in macaques, human lOFC and mOFC/vmPFC are necessary for contingent learning and value-guided decision-making, respectively.SIGNIFICANCE STATEMENT The lateral and medial regions of the orbitofrontal cortex are cytoarchitectonically distinct and have different anatomical connections. Previous investigations in macaques have shown these anatomical differences are accompanied by functional specialization for learning and decision-making. Here, for the first time, we test the predictions made by macaque studies in an experiment with humans with frontal lobe lesions, asking whether behavioral impairments can be linked to lateral or medial orbitofrontal cortex. Using equivalent tasks and computational analyses, our findings broadly replicate the pattern reported after selective lesions in monkeys. Patients with lateral orbitofrontal damage had impaired credit assignment, whereas damage to medial orbitofrontal cortex meant that patients were more likely to be distracted by irrelevant options.

Connectivity reveals relationship of brain areas for reward-guided learning and decision making in human and monkey frontal cortex.

Reward-guided decision-making depends on a network of brain regions. Among these are the orbitofrontal and the anterior cingulate cortex. However, it is difficult to ascertain if these areas constitute anatomical and functional unities, and how these areas correspond between monkeys and humans. To address these questions we looked at connectivity profiles of these areas using resting-state functional MRI in 38 humans and 25 macaque monkeys. We sought brain regions in the macaque that resembled 10 human areas identified with decision making and brain regions in the human that resembled six macaque areas identified with decision making. We also used diffusion-weighted MRI to delineate key human orbital and medial frontal brain regions. We identified 21 different regions, many of which could be linked to particular aspects of reward-guided learning, valuation, and decision making, and in many cases we identified areas in the macaque with similar coupling profiles.

Using diffusion imaging to study human connectional anatomy.

Diffusion imaging can be used to estimate the routes taken by fiber pathways connecting different regions of the living brain. This approach has already supplied novel insights into in vivo human brain anatomy. For example, by detecting where connection patterns change, one can define anatomical borders between cortical regions or subcortical nuclei in the living human brain for the first time. Because diffusion tractography is a relatively new technique, however, it is important to assess its validity critically. We discuss the degree to which diffusion tractography meets the requirements of a technique to assess structural connectivity and how its results compare to those from the gold-standard tract tracing methods in nonhuman animals. We conclude that although tractography offers novel opportunities it also raises significant challenges to be addressed by further validation studies to define precisely the limitations and scope of this exciting new technique.

A neural circuit covarying with social hierarchy in macaques.

Despite widespread interest in social dominance, little is known of its neural correlates in primates. We hypothesized that social status in primates might be related to individual variation in subcortical brain regions implicated in other aspects of social and emotional behavior in other mammals. To examine this possibility we used magnetic resonance imaging (MRI), which affords the taking of quantitative measurements noninvasively, both of brain structure and of brain function, across many regions simultaneously. We carried out a series of tests of structural and functional MRI (fMRI) data in 25 group-living macaques. First, a deformation-based morphometric (DBM) approach was used to show that gray matter in the amygdala, brainstem in the vicinity of the raphe nucleus, and reticular formation, hypothalamus, and septum/striatum of the left hemisphere was correlated with social status. Second, similar correlations were found in the same areas in the other hemisphere. Third, similar correlations were found in a second data set acquired several months later from a subset of the same animals. Fourth, the strength of coupling between fMRI-measured activity in the same areas was correlated with social status. The network of subcortical areas, however, had no relationship with the sizes of individuals' social networks, suggesting the areas had a simple and direct relationship with social status. By contrast a second circuit in cortex, comprising the midsuperior temporal sulcus and anterior and dorsal prefrontal cortex, covaried with both individuals' social statuses and the social network sizes they experienced. This cortical circuit may be linked to the social cognitive processes that are taxed by life in more complex social networks and that must also be used if an animal is to achieve a high social status.

The Good, the Bad, and the Irrelevant: Neural Mechanisms of Learning Real and Hypothetical Rewards and Effort.

Natural environments are complex, and a single choice can lead to multiple outcomes. Agents should learn which outcomes are due to their choices and therefore relevant for future decisions and which are stochastic in ways common to all choices and therefore irrelevant for future decisions between options. We designed an experiment in which human participants learned the varying reward and effort magnitudes of two options and repeatedly chose between them. The reward associated with a choice was randomly real or hypothetical (i.e., participants only sometimes received the reward magnitude associated with the chosen option). The real/hypothetical nature of the reward on any one trial was, however, irrelevant for learning the longer-term values of the choices, and participants ought to have only focused on the informational content of the outcome and disregarded whether it was a real or hypothetical reward. However, we found that participants showed an irrational choice bias, preferring choices that had previously led, by chance, to a real reward in the last trial. Amygdala and ventromedial prefrontal activity was related to the way in which participants' choices were biased by real reward receipt. By contrast, activity in dorsal anterior cingulate cortex, frontal operculum/anterior insula, and especially lateral anterior prefrontal cortex was related to the degree to which participants resisted this bias and chose effectively in a manner guided by aspects of outcomes that had real and more sustained relationships with particular choices, suppressing irrelevant reward information for more optimal learning and decision making. SIGNIFICANCE STATEMENT: In complex natural environments, a single choice can lead to multiple outcomes. Human agents should only learn from outcomes that are due to their choices, not from outcomes without such a relationship. We designed an experiment to measure learning about reward and effort magnitudes in an environment in which other features of the outcome were random and had no relationship with choice. We found that, although people could learn about reward magnitudes, they nevertheless were irrationally biased toward repeating certain choices as a function of the presence or absence of random reward features. Activity in different brain regions in the prefrontal cortex either reflected the bias or reflected resistance to the bias.

Brain systems for probabilistic and dynamic prediction: computational specificity and integration.

A computational approach to functional specialization suggests that brain systems can be characterized in terms of the types of computations they perform, rather than their sensory or behavioral domains. We contrasted the neural systems associated with two computationally distinct forms of predictive model: a reinforcement-learning model of the environment obtained through experience with discrete events, and continuous dynamic forward modeling. By manipulating the precision with which each type of prediction could be used, we caused participants to shift computational strategies within a single spatial prediction task. Hence (using fMRI) we showed that activity in two brain systems (typically associated with reward learning and motor control) could be dissociated in terms of the forms of computations that were performed there, even when both systems were used to make parallel predictions of the same event. A region in parietal cortex, which was sensitive to the divergence between the predictions of the models and anatomically connected to both computational networks, is proposed to mediate integration of the two predictive modes to produce a single behavioral output.

Connectivity profiles reveal the relationship between brain areas for social cognition in human and monkey temporoparietal cortex.

The human ability to infer the thoughts and beliefs of others, often referred to as "theory of mind," as well as the predisposition to even consider others, are associated with activity in the temporoparietal junction (TPJ) area. Unlike the case of most human brain areas, we have little sense of whether or how TPJ is related to brain areas in other nonhuman primates. It is not possible to address this question by looking for similar task-related activations in nonhuman primates because there is no evidence that nonhuman primates engage in theory-of-mind tasks in the same manner as humans. Here, instead, we explore the relationship by searching for areas in the macaque brain that interact with other macaque brain regions in the same manner as human TPJ interacts with other human brain regions. In other words, we look for brain regions with similar positions within a distributed neural circuit in the two species. We exploited the fact that human TPJ has a unique functional connectivity profile with cortical areas with known homologs in the macaque. For each voxel in the macaque temporal and parietal cortex we evaluated the similarity of its functional connectivity profile to that of human TPJ. We found that areas in the middle part of the superior temporal cortex, often associated with the processing of faces and other social stimuli, have the most similar connectivity profile. These results suggest that macaque face processing areas and human mentalizing areas might have a similar precursor.