Activation and disruption of a neural mechanism for novel choice in monkeys.

Neural mechanisms that mediate the ability to make value-guided decisions have received substantial attention in humans and animals1-6. Experiments in animals typically involve long training periods. By contrast, choices in the real world often need to be made between new options spontaneously. It is therefore possible that the neural mechanisms targeted in animal studies differ from those required for new decisions, which are typical of human imaging studies. Here we show that the primate medial frontal cortex (MFC)7 is involved in making new inferential choices when the options have not been previously experienced. Macaques spontaneously inferred the values of new options via similarities with the component parts of previously encountered options. Functional magnetic resonance imaging (fMRI) suggested that this ability was mediated by the MFC, which is rarely investigated in monkeys3; MFC activity reflected different processes of comparison for unfamiliar and familiar options. Multidimensional representations of options in the MFC used a coding scheme resembling that of grid cells, which is well known in spatial navigation8,9, to integrate dimensions in this non-physical space10 during novel decision-making. By contrast, the orbitofrontal cortex held specific object-based value representations1,11. In addition, minimally invasive ultrasonic disruption12 of MFC, but not adjacent tissue, altered the estimation of novel choice values.

Increasing and decreasing interregional brain coupling increases and decreases oscillatory activity in the human brain.

The origins of oscillatory activity in the brain are currently debated, but common to many hypotheses is the notion that they reflect interactions between brain areas. Here, we examine this possibility by manipulating the strength of coupling between two human brain regions, ventral premotor cortex (PMv) and primary motor cortex (M1), and examine the impact on oscillatory activity in the motor system measurable in the electroencephalogram. We either increased or decreased the strength of coupling while holding the impact on each component area in the pathway constant. This was achieved by stimulating PMv and M1 with paired pulses of transcranial magnetic stimulation using two different patterns, only one of which increases the influence exerted by PMv over M1. While the stimulation protocols differed in their temporal patterning, they were comprised of identical numbers of pulses to M1 and PMv. We measured the impact on activity in alpha, beta, and theta bands during a motor task in which participants either made a preprepared action (Go) or withheld it (No-Go). Augmenting cortical connectivity between PMv and M1, by evoking synchronous pre- and postsynaptic activity in the PMv-M1 pathway, enhanced oscillatory beta and theta rhythms in Go and No-Go trials, respectively. Little change was observed in the alpha rhythm. By contrast, diminishing the influence of PMv over M1 decreased oscillatory beta and theta rhythms in Go and No-Go trials, respectively. This suggests that corticocortical communication frequencies in the PMv-M1 pathway can be manipulated following Hebbian spike-timing-dependent plasticity.

Model-free decision making is prioritized when learning to avoid harming others.

Moral behavior requires learning how our actions help or harm others. Theoretical accounts of learning propose a key division between "model-free" algorithms that cache outcome values in actions and "model-based" algorithms that map actions to outcomes. Here, we tested the engagement of these mechanisms and their neural basis as participants learned to avoid painful electric shocks for themselves and a stranger. We found that model-free decision making was prioritized when learning to avoid harming others compared to oneself. Model-free prediction errors for others relative to self were tracked in the thalamus/caudate. At the time of choice, neural activity consistent with model-free moral learning was observed in subgenual anterior cingulate cortex (sgACC), and switching after harming others was associated with stronger connectivity between sgACC and dorsolateral prefrontal cortex. Finally, model-free moral learning varied with individual differences in moral judgment. Our findings suggest moral learning favors efficiency over flexibility and is underpinned by specific neural mechanisms.

Structural and resting state functional connectivity beyond the cortex.

Mapping the structural and functional connectivity of the central nervous system has become a key area within neuroimaging research. While detailed network structures across the entire brain have been probed using animal models, non-invasive neuroimaging in humans has thus far been dominated by cortical investigations. Beyond the cortex, subcortical nuclei have traditionally been less accessible due to their smaller size and greater distance from radio frequency coils. However, major neuroimaging developments now provide improved signal and the resolution required to study these structures. Here, we present an overview of the connectivity between the amygdala, brainstem, cerebellum, spinal cord and the rest of the brain. While limitations to their imaging and analyses remain, we also provide some recommendations and considerations for mapping brain connectivity beyond the cortex.

Neural Signatures of Value Comparison in Human Cingulate Cortex during Decisions Requiring an Effort-Reward Trade-off.

UNLABELLED: Integrating costs and benefits is crucial for optimal decision-making. Although much is known about decisions that involve outcome-related costs (e.g., delay, risk), many of our choices are attached to actions and require an evaluation of the associated motor costs. Yet how the brain incorporates motor costs into choices remains largely unclear. We used human fMRI during choices involving monetary reward and physical effort to identify brain regions that serve as a choice comparator for effort-reward trade-offs. By independently varying both options' effort and reward levels, we were able to identify the neural signature of a comparator mechanism. A network involving supplementary motor area and the caudal portion of dorsal anterior cingulate cortex encoded the difference in reward (positively) and effort levels (negatively) between chosen and unchosen choice options. We next modeled effort-discounted subjective values using a novel behavioral model. This revealed that the same network of regions involving dorsal anterior cingulate cortex and supplementary motor area encoded the difference between the chosen and unchosen options' subjective values, and that activity was best described using a concave model of effort-discounting. In addition, this signal reflected how precisely value determined participants' choices. By contrast, separate signals in supplementary motor area and ventromedial prefrontal cortex correlated with participants' tendency to avoid effort and seek reward, respectively. This suggests that the critical neural signature of decision-making for choices involving motor costs is found in human cingulate cortex and not ventromedial prefrontal cortex as typically reported for outcome-based choice. Furthermore, distinct frontal circuits seem to drive behavior toward reward maximization and effort minimization. SIGNIFICANCE STATEMENT: The neural processes that govern the trade-off between expected benefits and motor costs remain largely unknown. This is striking because energetic requirements play an integral role in our day-to-day choices and instrumental behavior, and a diminished willingness to exert effort is a characteristic feature of a range of neurological disorders. We use a new behavioral characterization of how humans trade off reward maximization with effort minimization to examine the neural signatures that underpin such choices, using BOLD MRI neuroimaging data. We find the critical neural signature of decision-making, a signal that reflects the comparison of value between choice options, in human cingulate cortex, whereas two distinct brain circuits drive behavior toward reward maximization or effort minimization.

Behavioral modeling of human choices reveals dissociable effects of physical effort and temporal delay on reward devaluation.

There has been considerable interest from the fields of biology, economics, psychology, and ecology about how decision costs decrease the value of rewarding outcomes. For example, formal descriptions of how reward value changes with increasing temporal delays allow for quantifying individual decision preferences, as in animal species populating different habitats, or normal and clinical human populations. Strikingly, it remains largely unclear how humans evaluate rewards when these are tied to energetic costs, despite the surge of interest in the neural basis of effort-guided decision-making and the prevalence of disorders showing a diminished willingness to exert effort (e.g., depression). One common assumption is that effort discounts reward in a similar way to delay. Here we challenge this assumption by formally comparing competing hypotheses about effort and delay discounting. We used a design specifically optimized to compare discounting behavior for both effort and delay over a wide range of decision costs (Experiment 1). We then additionally characterized the profile of effort discounting free of model assumptions (Experiment 2). Contrary to previous reports, in both experiments effort costs devalued reward in a manner opposite to delay, with small devaluations for lower efforts, and progressively larger devaluations for higher effort-levels (concave shape). Bayesian model comparison confirmed that delay-choices were best predicted by a hyperbolic model, with the largest reward devaluations occurring at shorter delays. In contrast, an altogether different relationship was observed for effort-choices, which were best described by a model of inverse sigmoidal shape that is initially concave. Our results provide a novel characterization of human effort discounting behavior and its first dissociation from delay discounting. This enables accurate modelling of cost-benefit decisions, a prerequisite for the investigation of the neural underpinnings of effort-guided choice and for understanding the deficits in clinical disorders characterized by behavioral inactivity.

Variability of human corticospinal excitability tracks the state of action preparation.

Task-evoked trial-by-trial variability is a ubiquitous property of neural responses, yet its functional role remains largely unclear. Recent work in nonhuman primates shows that the temporal structure of neural variability in several brain regions is task-related. For example, trial-by-trial variability in premotor cortex tracks motor preparation with increasingly consistent firing rates and thus a decline in variability before movement onset. However, whether noninvasive measures of the variability of population activity available from humans can similarly track the preparation of actions remains unknown. We tested this by using single-pulse transcranial magnetic stimulation (TMS) over primary motor cortex (M1) to measure corticospinal excitability (CSE) at different times during action preparation. First, we established the basic properties of intrinsic CSE variability at rest. Then, during the task, responses (left or right button presses) were either directly instructed (forced choice) or resulted from a value decision (choice). Before movement onset, we observed a temporally specific task-related decline in CSE variability contralateral to the responding hand. This decline was stronger in fast-response compared with slow-response trials, consistent with data in nonhuman primates. For the nonresponding hand, CSE variability also decreased, but only in choice trials, and earlier compared with the responding hand, possibly reflecting choice-specific suppression of unselected actions. These findings suggest that human CSE variability measured by TMS over M1 tracks the state of motor preparation, and may reflect the optimization of preparatory population activity. This provides novel avenues in humans to assess the dynamics of action preparation but also more complex processes, such as choice-to-action transformations.

Nuclei-specific hypothalamus networks predict a dimensional marker of stress in humans.

The hypothalamus is part of the hypothalamic-pituitary-adrenal axis which activates stress responses through release of cortisol. It is a small but heterogeneous structure comprising multiple nuclei. In vivo human neuroimaging has rarely succeeded in recording signals from individual hypothalamus nuclei. Here we use human resting-state fMRI (n = 498) with high spatial resolution to examine relationships between the functional connectivity of specific hypothalamic nuclei and a dimensional marker of prolonged stress. First, we demonstrate that we can parcellate the human hypothalamus into seven nuclei in vivo. Using the functional connectivity between these nuclei and other subcortical structures including the amygdala, we significantly predict stress scores out-of-sample. Predictions use 0.0015% of all possible brain edges, are specific to stress, and improve when using nucleus-specific compared to whole-hypothalamus connectivity. Thus, stress relates to connectivity changes in precise and functionally meaningful subcortical networks, which may be exploited in future studies using interventions in stress disorders.

Oxidative phosphorylation, not glycolysis, powers presynaptic and postsynaptic mechanisms underlying brain information processing.

Neural activity has been suggested to initially trigger ATP production by glycolysis, rather than oxidative phosphorylation, for three reasons: glycolytic enzymes are associated with ion pumps; neurons may increase their energy supply by activating glycolysis in astrocytes to generate lactate; and activity increases glucose uptake more than O2 uptake. In rat hippocampal slices, neuronal activity rapidly decreased the levels of extracellular O2 and intracellular NADH (reduced nicotinamide adenine dinucleotide), even with lactate dehydrogenase blocked to prevent lactate generation, or with only 20% superfused O2 to mimic physiological O2 levels. Pharmacological analysis revealed an energy budget in which 11% of O2 use was on presynaptic action potentials, 17% was on presynaptic Ca²âº entry and transmitter release, 46% was on postsynaptic glutamate receptors, and 26% was on postsynaptic action potentials, in approximate accord with theoretical brain energy budgets. Thus, the major mechanisms mediating brain information processing are all initially powered by oxidative phosphorylation, and an astrocyte-neuron lactate shuttle is not needed for this to occur.

The association of longitudinal diet and waist-to-hip ratio from midlife to old age with hippocampus connectivity and memory in old age: a cohort study.

Epidemiological studies suggest lifestyle factors may reduce the risk of dementia. However, few studies have examined the association of diet and waist-to-hip ratio with hippocampus connectivity. In the Whitehall II Imaging Sub-study, we examined longitudinal changes in diet quality in 512 participants and waist-to-hip ratio in 665 participants. Diet quality was measured using the Alternative Health Eating Index-2010 assessed three times across 11 years between ages 48 and 60 years, and waist-to-hip ratio five times over 21 years between ages 48 and 68 years. Brain imaging and cognitive tests were performed at age 70±5 years. We measured white matter using diffusion tensor imaging and hippocampal functional connectivity using resting-state functional magnetic resonance imaging. In addition to associations of diet and waist-to-hip ratio with brain imaging measures, we tested whether associations between diet, waist-to-hip ratio and cognitive performance were mediated by brain connectivity. We found better diet quality in midlife and improvements in diet over mid-to-late life were associated with higher hippocampal functional connectivity to the occipital lobe and cerebellum, and better white matter integrity as measured by higher fractional anisotropy and lower diffusivity. Higher waist-to-hip ratio in midlife was associated with higher mean and radial diffusivity and lower fractional anisotropy in several tracts including the inferior longitudinal fasciculus and cingulum. Associations between midlife waist-to-hip ratio, working memory and executive function were partially mediated by radial diffusivity. All associations were independent of age, sex, education, and physical activity. Our findings highlight the importance of maintaining a good diet and a healthy waist-to-hip ratio in midlife to maintain brain health in later life. Future interventional studies for the improvement of dietary and metabolic health should target midlife for the prevention of cognitive decline in old age.

Action initiation and punishment learning differ from childhood to adolescence while reward learning remains stable.

Theoretical and empirical accounts suggest that adolescence is associated with heightened reward learning and impulsivity. Experimental tasks and computational models that can dissociate reward learning from the tendency to initiate actions impulsively (action initiation bias) are thus critical to characterise the mechanisms that drive developmental differences. However, existing work has rarely quantified both learning ability and action initiation, or it has relied on small samples. Here, using computational modelling of a learning task collected from a large sample (N = 742, 9-18 years, 11 countries), we test differences in reward and punishment learning and action initiation from childhood to adolescence. Computational modelling reveals that whilst punishment learning rates increase with age, reward learning remains stable. In parallel, action initiation biases decrease with age. Results are similar when considering pubertal stage instead of chronological age. We conclude that heightened reward responsivity in adolescence can reflect differences in action initiation rather than enhanced reward learning.

Medial and orbital frontal cortex in decision-making and flexible behavior.

The medial frontal cortex and adjacent orbitofrontal cortex have been the focus of investigations of decision-making, behavioral flexibility, and social behavior. We review studies conducted in humans, macaques, and rodents and argue that several regions with different functional roles can be identified in the dorsal anterior cingulate cortex, perigenual anterior cingulate cortex, anterior medial frontal cortex, ventromedial prefrontal cortex, and medial and lateral parts of the orbitofrontal cortex. There is increasing evidence that the manner in which these areas represent the value of the environment and specific choices is different from subcortical brain regions and more complex than previously thought. Although activity in some regions reflects distributions of reward and opportunities across the environment, in other cases, activity reflects the structural relationships between features of the environment that animals can use to infer what decision to take even if they have not encountered identical opportunities in the past.

Relationship between nuclei-specific amygdala connectivity and mental health dimensions in humans.

There has been increasing interest in using neuroimaging measures to predict psychiatric disorders. However, predictions usually rely on large brain networks and large disorder heterogeneity. Thus, they lack both anatomical and behavioural specificity, preventing the advancement of targeted interventions. Here we address both challenges. First, using resting-state functional magnetic resonance imaging, we parcellated the amygdala, a region implicated in mood disorders, into seven nuclei. Next, a questionnaire factor analysis provided subclinical mental health dimensions frequently altered in anxious-depressive individuals, such as negative emotions and sleep problems. Finally, for each behavioural dimension, we identified the most predictive resting-state functional connectivity between individual amygdala nuclei and highly specific regions of interest, such as the dorsal raphe nucleus in the brainstem or medial frontal cortical regions. Connectivity in circumscribed amygdala networks predicted behaviours in an independent dataset. Our results reveal specific relations between mental health dimensions and connectivity in precise subcortical networks.

Computational modelling of social cognition and behaviour-a reinforcement learning primer.

Social neuroscience aims to describe the neural systems that underpin social cognition and behaviour. Over the past decade, researchers have begun to combine computational models with neuroimaging to link social computations to the brain. Inspired by approaches from reinforcement learning theory, which describes how decisions are driven by the unexpectedness of outcomes, accounts of the neural basis of prosocial learning, observational learning, mentalizing and impression formation have been developed. Here we provide an introduction for researchers who wish to use these models in their studies. We consider both theoretical and practical issues related to their implementation, with a focus on specific examples from the field.

Neural and computational mechanisms of momentary fatigue and persistence in effort-based choice.

From a gym workout, to deciding whether to persevere at work, many activities require us to persist in deciding that rewards are 'worth the effort' even as we become fatigued. However, studies examining effort-based decisions typically assume that the willingness to work is static. Here, we use computational modelling on two effort-based tasks, one behavioural and one during fMRI. We show that two hidden states of fatigue fluctuate on a moment-to-moment basis on different timescales but both reduce the willingness to exert effort for reward. The value of one state increases after effort but is 'recoverable' by rests, whereas a second 'unrecoverable' state gradually increases with work. The BOLD response in separate medial and lateral frontal sub-regions covaried with these states when making effort-based decisions, while a distinct fronto-striatal system integrated fatigue with value. These results provide a computational framework for understanding the brain mechanisms of persistence and momentary fatigue.

Adapting non-invasive human recordings along multiple task-axes shows unfolding of spontaneous and over-trained choice.

Choices rely on a transformation of sensory inputs into motor responses. Using invasive single neuron recordings, the evolution of a choice process has been tracked by projecting population neural responses into state spaces. Here, we develop an approach that allows us to recover similar trajectories on a millisecond timescale in non-invasive human recordings. We selectively suppress activity related to three task-axes, relevant and irrelevant sensory inputs and response direction, in magnetoencephalography data acquired during context-dependent choices. Recordings from premotor cortex show a progression from processing sensory input to processing the response. In contrast to previous macaque recordings, information related to choice-irrelevant features is represented more weakly than choice-relevant sensory information. To test whether this mechanistic difference between species is caused by extensive over-training common in non-human primate studies, we trained humans on >20,000 trials of the task. Choice-irrelevant features were still weaker than relevant features in premotor cortex after over-training.

Associations of dietary markers with brain volume and connectivity: A systematic review of MRI studies.

The high prevalence of unhealthy dietary patterns and related brain disorders, such as dementia, emphasizes the importance of research that examines the effect of dietary factors on brain health. Identifying markers of brain health, such as volume and connectivity, that relate to diet is an important first step towards understanding the lifestyle determinants of healthy brain ageing. We conducted a systematic review of 52 studies (total n = 21,221 healthy participants aged 26-80 years, 55 % female) that assessed with a range of MRI measurements, which brain areas, connections, and cerebrovascular factors were associated with dietary markers. We report associations between regional brain measures and dietary health. Collectively, lower diet quality was related to reduced brain volume and connectivity, especially in white and grey matter of the frontal, temporal and parietal lobe, cingulate, entorhinal cortex and the hippocampus. Associations were also observed in connecting fibre pathways and in particular the default-mode, sensorimotor and attention networks. However, there were also some inconsistencies in research methods and findings. We recommend that future research use more comprehensive and consistent dietary measures, more representative samples, and examine the role of key subcortical regions previously highlighted in relevant animal work.

Polarity of uncertainty representation during exploration and exploitation in ventromedial prefrontal cortex.

Environments furnish multiple information sources for making predictions about future events. Here we use behavioural modelling and functional magnetic resonance imaging to describe how humans select predictors that might be most relevant. First, during early encounters with potential predictors, participants' selections were explorative and directed towards subjectively uncertain predictors (positive uncertainty effect). This was particularly the case when many future opportunities remained to exploit knowledge gained. Then, preferences for accurate predictors increased over time, while uncertain predictors were avoided (negative uncertainty effect). The behavioural transition from positive to negative uncertainty-driven selections was accompanied by changes in the representations of belief uncertainty in ventromedial prefrontal cortex (vmPFC). The polarity of uncertainty representations (positive or negative encoding of uncertainty) changed between exploration and exploitation periods. Moreover, the two periods were separated by a third transitional period in which beliefs about predictors' accuracy predominated. The vmPFC signals a multiplicity of decision variables, the strength and polarity of which vary with behavioural context.

Causal manipulation of self-other mergence in the dorsomedial prefrontal cortex.

To navigate social environments, people must simultaneously hold representations about their own and others' abilities. During self-other mergence, people estimate others' abilities not only on the basis of the others' past performance, but the estimates are also influenced by their own performance. For example, if we perform well, we overestimate the abilities of those with whom we are co-operating and underestimate competitors. Self-other mergence is associated with specific activity patterns in the dorsomedial prefrontal cortex (dmPFC). Using a combination of non-invasive brain stimulation, functional magnetic resonance imaging, and computational modeling, we show that dmPFC neurostimulation silences these neural signatures of self-other mergence in relation to estimation of others' abilities. In consequence, self-other mergence behavior increases, and our assessments of our own performance are projected increasingly onto other people. This suggests an inherent tendency to form interdependent social representations and a causal role of the dmPFC in separating self and other representations.

Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys.

People and other animals learn the values of choices by observing the contingencies between them and their outcomes. However, decisions are not guided by choice-linked reward associations alone; macaques also maintain a memory of the general, average reward rate - the global reward state - in an environment. Remarkably, global reward state affects the way that each choice outcome is valued and influences future decisions so that the impact of both choice success and failure is different in rich and poor environments. Successful choices are more likely to be repeated but this is especially the case in rich environments. Unsuccessful choices are more likely to be abandoned but this is especially likely in poor environments. Functional magnetic resonance imaging (fMRI) revealed two distinct patterns of activity, one in anterior insula and one in the dorsal raphe nucleus, that track global reward state as well as specific outcome events.