Acetylcholine and noradrenaline enhance foraging optimality in humans.

Foraging theory prescribes when optimal foragers should leave the current option for more rewarding alternatives. Actual foragers often exploit options longer than prescribed by the theory, but it is unclear how this foraging suboptimality arises. We investigated whether the upregulation of cholinergic, noradrenergic, and dopaminergic systems increases foraging optimality. In a double-blind, between-subject design, participants (N = 160) received placebo, the nicotinic acetylcholine receptor agonist nicotine, a noradrenaline reuptake inhibitor reboxetine, or a preferential dopamine reuptake inhibitor methylphenidate, and played the role of a farmer who collected milk from patches with different yield. Across all groups, participants on average overharvested. While methylphenidate had no effects on this bias, nicotine, and to some extent also reboxetine, significantly reduced deviation from foraging optimality, which resulted in better performance compared to placebo. Concurring with amplified goal-directedness and excluding heuristic explanations, nicotine independently also improved trial initiation and time perception. Our findings elucidate the neurochemical basis of behavioral flexibility and decision optimality and open unique perspectives on psychiatric disorders affecting these functions.

The nematode worm C. elegans chooses between bacterial foods as if maximizing economic utility.

In value-based decision making, options are selected according to subjective values assigned by the individual to available goods and actions. Despite the importance of this faculty of the mind, the neural mechanisms of value assignments, and how choices are directed by them, remain obscure. To investigate this problem, we used a classic measure of utility maximization, the Generalized Axiom of Revealed Preference, to quantify internal consistency of food preferences in Caenorhabditis elegans, a nematode worm with a nervous system of only 302 neurons. Using a novel combination of microfluidics and electrophysiology, we found that C. elegans food choices fulfill the necessary and sufficient conditions for utility maximization, indicating that nematodes behave as if they maintain, and attempt to maximize, an underlying representation of subjective value. Food choices are well-fit by a utility function widely used to model human consumers. Moreover, as in many other animals, subjective values in C. elegans are learned, a process we find requires intact dopamine signaling. Differential responses of identified chemosensory neurons to foods with distinct growth potentials are amplified by prior consumption of these foods, suggesting that these neurons may be part of a value-assignment system. The demonstration of utility maximization in an organism with a very small nervous system sets a new lower bound on the computational requirements for utility maximization and offers the prospect of an essentially complete explanation of value-based decision making at single neuron resolution in this organism.

Increased random exploration in schizophrenia is associated with inflammation.

One aspect of goal-directed behavior, which is known to be impaired in patients with schizophrenia (SZ), is balancing between exploiting a familiar choice with known reward value and exploring a lesser known, but potentially more rewarding option. Despite its relevance to several symptom domains of SZ, this has received little attention in SZ research. In addition, while there is increasing evidence that SZ is associated with chronic low-grade inflammation, few studies have investigated how this relates to specific behaviors, such as balancing exploration and exploitation. We therefore assessed behaviors underlying the exploration-exploitation trade-off using a three-armed bandit task in 45 patients with SZ and 19 healthy controls (HC). This task allowed us to dissociate goal-unrelated (random) from goal-related (directed) exploration and correlate them with psychopathological symptoms. Moreover, we assessed a broad range of inflammatory proteins in the blood and related them to bandit task behavior. We found that, compared to HC, patients with SZ showed reduced task performance. This impairment was due to a shift from exploitation to random exploration, which was associated with symptoms of disorganization. Relative to HC, patients with SZ showed a pro-inflammatory blood profile. Furthermore, high-sensitivity C-reactive protein (hsCRP) positively correlated with random exploration, but not with directed exploration or exploitation. In conclusion, we show that low-grade inflammation in patients with SZ is associated with random exploration, which can be considered a behavioral marker for disorganization. hsCRP may constitute a marker for severity of, and a potential treatment target for maladaptive exploratory behaviors.

Conditional valuation for combinations of goods in primates.

Valuing goods and selecting the one with the highest value forms the basis of adaptive behaviour across species. While it is obvious that the valuation of a given type of goods depends on ownership and availability of that type of goods, the effects of other goods on valuation of the original good are sometimes underappreciated. Yet, goods interact with each other, indicating that the valuation of a given good is conditional on the other goods it is combined with, both in the wild and the laboratory. Here, we introduce conditional valuation in the context of valuing multiple goods and briefly review how human and animal experimentalists can leverage economic tools for the study of interactions among goods. We then review evidence for conditional valuation for combined foods in both human and non-human primates. In the laboratory, non-human primates show increased valuation of certain combinations of foods but decreased valuation of other types of combinations. Thus, similarly to humans, monkeys appear to value combinations of goods in a conditional fashion. Additionally, both humans and monkeys appear to employ similar neural substrates for the valuation of single goods, such as the orbitofrontal cortex. Together, investigations of our evolutionary precursors may provide insights on how we value interacting goods. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Sensitivity of reaction time to the magnitude of rewards reveals the cost-structure of time.

The Drift-Diffusion Model (DDM) is the prevalent computational model of the speed-accuracy trade-off in decision making. The DDM provides an explanation of behavior by optimally balancing reaction times and error rates. However, when applied to value-based decision making, the DDM makes the stark prediction that reaction times depend only on the relative utility difference between the options and not on absolute utility magnitudes. This prediction runs counter to evidence that reaction times decrease with higher utility magnitude. Here, we ask if and how it could be optimal for reaction times to show this observed pattern. We study an algorithmic framework that balances the cost of delaying rewards against the utility of obtained rewards. We find that the functional form of the cost of delay plays a key role, with the empirically observed pattern becoming optimal under multiplicative discounting. We add to the empirical literature by testing whether utility magnitude affects reaction times using a novel methodology that does not rely on functional form assumptions for the subjects' utilities. Our results advance the understanding of how and why reaction times are sensitive to the magnitude of rewards.

Why Do Irrelevant Alternatives Matter? An fMRI-TMS Study of Context-Dependent Preferences.

Both humans and animals are known to exhibit a violation of rationality known as "decoy effect": introducing an irrelevant option (a decoy) can influence choices among other (relevant) options. Exactly how and why decoys trigger this effect is not known. It may be an example of fast heuristic decision-making, which is adaptive in natural environments, but may lead to biased choices in certain markets or experiments. We used fMRI and transcranial magnetic stimulation to investigate the neural underpinning of the decoy effect of both sexes. The left ventral striatum was more active when the chosen option dominated the decoy. This is consistent with the hypothesis that the presence of a decoy option influences the valuation of other options, making valuation context-dependent even when choices appear fully rational. Consistent with the idea that control is recruited to prevent heuristics from producing biased choices, the right inferior frontal gyrus, often implicated in inhibiting prepotent responses, connected more strongly with the striatum when subjects successfully overrode the decoy effect and made unbiased choices. This is further supported by our transcranial magnetic stimulation experiment: subjects whose right inferior frontal gyrus was temporarily disrupted made biased choices more often than a control group. Our results suggest that the neural basis of the decoy effect could be the context-dependent activation of the valuation area. But the differential connectivity from the frontal area may indicate how deliberate control monitors and corrects errors and biases in decision-making.SIGNIFICANCE STATEMENT Standard theories of rational decision-making assume context-independent valuations of available options. Motivated by the importance of this basic assumption, we used fMRI to study how the human brain assigns values to available options. We found activity in the valuation area to be consistent with the hypothesis that values depend on irrelevant aspects of the environment, even for subjects whose choices appear fully rational. Such context-dependent valuations may lead to biased decision-making. We further found differential connectivity from the frontal area to the valuation area depending on whether biases were successfully overcome. This suggests a mechanism for making rational choices despite the potential bias. Further support was obtained by a transcranial magnetic stimulation experiment, where subjects whose frontal control was temporarily disrupted made biased choices more often than a control group.

The Reduction of Ventrolateral Prefrontal Cortex Gray Matter Volume Correlates with Loss of Economic Rationality in Aging.

The population of people above 65 years old continues to grow, and there is mounting evidence that as humans age they are more likely to make errors. However, the specific effect of neuroanatomical aging on the efficiency of economic decision-making is poorly understood. We used whole-brain voxel-based morphometry analysis to determine where reduction of gray matter volume in healthy female and male adults over the age of 65 years correlates with a classic measure of economic irrationality: violations of the Generalized Axiom of Revealed Preference. All participants were functionally normal with Mini-Mental State Examination scores ranging between 26 and 30. While our elders showed the previously reported decline in rationality compared with younger subjects, chronological age per se did not correlate with rationality measures within our population of elders. Instead, reduction of gray matter density in ventrolateral prefrontal cortex correlates tightly with irrational behavior. Interestingly, using a large fMRI sample and meta-analytic tool with Neurosynth, we found that this brain area shows strong coactivation patterns with nearly all of the value-associated regions identified in previous studies. These findings point toward a neuroanatomic locus for economic rationality in the aging brain and highlight the importance of understanding both anatomy and function in the study of aging, cognition, and decision-making.SIGNIFICANCE STATEMENT Age is a crucial factor in decision-making, with older individuals making more errors in choices. Using whole-brain voxel-based morphometry analysis, we found that reduction of gray matter density in ventrolateral prefrontal cortex correlates with economic irrationality: reduced gray matter volume in this area correlates with the frequency and severity of violations of the Generalized Axiom of Revealed Preference. Furthermore, this brain area strongly coactivates with other reward-associated regions identified with Neurosynth. These findings point toward a role for neuroscientific discoveries in shaping long-standing economic views of decision-making.

The interaction between expected values and risk levels in a modified Iowa gambling task.

Performance on the Iowa gambling task (IGT) supports somatic marker hypothesis (SMH), which proposes that the process of decision making depends on emotion (Damasio, 1994). However, the bad decks in the IGT are also more risky and that confounds the results. To resolve this issue, the IGT-Yen, a variant of the IGT, was created to independently examine the effects of expected value and risk. After 20 trials, participants selected more high-risk bad decks than low-risk bad decks and more low-risk good decks than high-risk good decks. Greater anticipatory skin conductance levels (SCLs) were associated with choosing high-risk bad decks compared to choosing low-risk bad decks in trials 21-80, and greater anticipatory SCLs were associated with choosing low-risk good decks compared to choosing high-risk good decks in trials 81-100. Therefore, the anticipatory SCLs were associated with expected values of the decks and with their levels of risk.