Quantifying the subjective cost of self-control in humans.

Since Odysseus committed to resisting the Sirens, mechanisms to limit self-control failure have been a central feature of human behavior. Psychologists have long argued that the use of self-control is an effortful process and, more recently, that its failure arises when the cognitive costs of self-control outweigh its perceived benefits. In a similar way, economists have argued that sophisticated choosers can adopt "precommitment strategies" that tie the hands of their future selves in order to reduce these costs. Yet, we still lack an empirical tool to quantify and demonstrate the cost of self-control. Here, we develop and validate an economic decision-making task to quantify the subjective cost of self-control by determining the monetary cost a person is willing to incur in order to eliminate the need for self-control. We find that humans will pay to avoid having to exert self-control in a way that scales with increasing levels of temptation and that these costs appear to be modulated both by motivational incentives and stress exposure. Our psychophysical approach allows us to index moment-to-moment self-control costs at the within-subject level, validating important theoretical work across multiple disciplines and opening avenues of self-control research in healthy and clinical populations.

The computational form of craving is a selective multiplication of economic value.

Craving is thought to be a specific desire state that biases choice toward the desired object, be it chocolate or drugs. A vast majority of people report having experienced craving of some kind. In its pathological form craving contributes to health outcomes in addiction and obesity. Yet despite its ubiquity and clinical relevance we still lack a basic neurocomputational understanding of craving. Here, using an instantaneous measure of subjective valuation and selective cue exposure, we identify a behavioral signature of a food craving-like state and advance a computational framework for understanding how this state might transform valuation to bias choice. We find desire induced by exposure to a specific high-calorie, high-fat/sugar snack good is expressed in subjects' momentary willingness to pay for this good. This effect is selective but not exclusive to the exposed good; rather, we find it generalizes to nonexposed goods in proportion to their subjective attribute similarity to the exposed ones. A second manipulation of reward size (number of snack units available for purchase) further suggested that a multiplicative gain mechanism supports the transformation of valuation during laboratory craving. These findings help explain how real-world food craving can result in behaviors inconsistent with preferences expressed in the absence of craving and open a path for the computational modeling of craving-like phenomena using a simple and repeatable experimental tool for assessing subjective states in economic terms.

Normalized value coding explains dynamic adaptation in the human valuation process.

The notion of subjective value is central to choice theories in ecology, economics, and psychology, serving as an integrated decision variable by which options are compared. Subjective value is often assumed to be an absolute quantity, determined in a static manner by the properties of an individual option. Recent neurobiological studies, however, have shown that neural value coding dynamically adapts to the statistics of the recent reward environment, introducing an intrinsic temporal context dependence into the neural representation of value. Whether valuation exhibits this kind of dynamic adaptation at the behavioral level is unknown. Here, we show that the valuation process in human subjects adapts to the history of previous values, with current valuations varying inversely with the average value of recently observed items. The dynamics of this adaptive valuation are captured by divisive normalization, linking these temporal context effects to spatial context effects in decision making as well as spatial and temporal context effects in perception. These findings suggest that adaptation is a universal feature of neural information processing and offer a unifying explanation for contextual phenomena in fields ranging from visual psychophysics to economic choice.

Modulation of Saccade Vigor during Value-Based Decision Making.

During value-based decision-making, individuals consider the various options and select the one that provides the maximum subjective value. Although the brain integrates abstract information to compute and compare these values, the only behavioral outcome is often the decision itself. However, if the options are visual stimuli, during deliberation the brain moves the eyes from one stimulus to the other. Previous work suggests that saccade vigor, i.e., peak velocity as a function of amplitude, is greater if reward is associated with the visual stimulus. This raises the possibility that vigor during the free viewing of options may be influenced by the valuation of each option. Here, humans chose between a small, immediate monetary reward and a larger but delayed reward. As the deliberation began, vigor was similar for the saccades made to the two options but diverged 0.5 s before decision time, becoming greater for the preferred option. This difference in vigor increased as a function of the difference in the subjective values that the participant assigned to the delayed and immediate options. After the decision was made, participants continued to gaze at the options, but with reduced vigor, making it possible to infer timing of the decision from the sudden drop in vigor. Therefore, the subjective value that the brain assigned to a stimulus during decision-making affected the motor system via the vigor with which the eyes moved toward that stimulus. SIGNIFICANCE STATEMENT: We find that, as individuals deliberate between two rewarding options and arrive at a decision, the vigor with which they make saccades to each option reflects a real-time evaluation of that option. With deliberation, saccade vigor diverges between the two options, becoming greater for the option that the individual will eventually choose. The results suggest a shared element between the network that assigns value to a stimulus during the process of decision-making and the network that controls vigor of movements toward that stimulus.

Unexpected but Incidental Positive Outcomes Predict Real-World Gambling.

Positive mood can affect a person's tendency to gamble, possibly because positive mood fosters unrealistic optimism. At the same time, unexpected positive outcomes, often called prediction errors, influence mood. However, a linkage between positive prediction errors-the difference between expected and obtained outcomes-and consequent risk taking has yet to be demonstrated. Using a large data set of New York City lottery gambling and a model inspired by computational accounts of reward learning, we found that people gamble more when incidental outcomes in the environment (e.g., local sporting events and sunshine) are better than expected. When local sports teams performed better than expected, or a sunny day followed a streak of cloudy days, residents gambled more. The observed relationship between prediction errors and gambling was ubiquitous across the city's socioeconomically diverse neighborhoods and was specific to sports and weather events occurring locally in New York City. Our results suggest that unexpected but incidental positive outcomes influence risk taking.

Thirst-dependent risk preferences in monkeys identify a primitive form of wealth.

Experimental economic techniques have been widely used to evaluate human risk attitudes, but how these measured attitudes relate to overall individual wealth levels is unclear. Previous noneconomic work has addressed this uncertainty in animals by asking the following: (i) Do our close evolutionary relatives share both our risk attitudes and our degree of economic rationality? And (ii) how does the amount of food or water one holds (a nonpecuniary form of "wealth") alter risk attitudes in these choosers? Unfortunately, existing noneconomic studies have provided conflicting insights from an economic point of view. We therefore used standard techniques from human experimental economics to measure monkey risk attitudes for water rewards as a function of blood osmolality (an objective measure of how much water the subjects possess). Early in training, monkeys behaved randomly, consistently violating first-order stochastic dominance and monotonicity. After training, they behaved like human choosers--technically consistent in their choices and weakly risk averse (i.e., risk averse or risk neutral on average)--suggesting that well-trained monkeys can serve as a model for human choice behavior. As with attitudes about money in humans, these risk attitudes were strongly wealth dependent; as the animals became "poorer," risk aversion increased, a finding incompatible with some models of wealth and risk in human decision making.

Normalization is a general neural mechanism for context-dependent decision making.

Understanding the neural code is critical to linking brain and behavior. In sensory systems, divisive normalization seems to be a canonical neural computation, observed in areas ranging from retina to cortex and mediating processes including contrast adaptation, surround suppression, visual attention, and multisensory integration. Recent electrophysiological studies have extended these insights beyond the sensory domain, demonstrating an analogous algorithm for the value signals that guide decision making, but the effects of normalization on choice behavior are unknown. Here, we show that choice models using normalization generate significant (and classically irrational) choice phenomena driven by either the value or number of alternative options. In value-guided choice experiments, both monkey and human choosers show novel context-dependent behavior consistent with normalization. These findings suggest that the neural mechanism of value coding critically influences stochastic choice behavior and provide a generalizable quantitative framework for examining context effects in decision making.

Like cognitive function, decision making across the life span shows profound age-related changes.

It has long been known that human cognitive function improves through young adulthood and then declines across the later life span. Here we examined how decision-making function changes across the life span by measuring risk and ambiguity attitudes in the gain and loss domains, as well as choice consistency, in an urban cohort ranging in age from 12 to 90 y. We identified several important age-related patterns in decision making under uncertainty: First, we found that healthy elders between the ages of 65 and 90 were strikingly inconsistent in their choices compared with younger subjects. Just as elders show profound declines in cognitive function, they also show profound declines in choice rationality compared with their younger peers. Second, we found that the widely documented phenomenon of ambiguity aversion is specific to the gain domain and does not occur in the loss domain, except for a slight effect in older adults. Finally, extending an earlier report by our group, we found that risk attitudes across the life span show an inverted U-shaped function; both elders and adolescents are more risk-averse than their midlife counterparts. Taken together, these characterizations of decision-making function across the life span in this urban cohort strengthen the conclusions of previous reports suggesting a profound impact of aging on cognitive function in this domain.

Dynamic divisive normalization predicts time-varying value coding in decision-related circuits.

Normalization is a widespread neural computation, mediating divisive gain control in sensory processing and implementing a context-dependent value code in decision-related frontal and parietal cortices. Although decision-making is a dynamic process with complex temporal characteristics, most models of normalization are time-independent and little is known about the dynamic interaction of normalization and choice. Here, we show that a simple differential equation model of normalization explains the characteristic phasic-sustained pattern of cortical decision activity and predicts specific normalization dynamics: value coding during initial transients, time-varying value modulation, and delayed onset of contextual information. Empirically, we observe these predicted dynamics in saccade-related neurons in monkey lateral intraparietal cortex. Furthermore, such models naturally incorporate a time-weighted average of past activity, implementing an intrinsic reference-dependence in value coding. These results suggest that a single network mechanism can explain both transient and sustained decision activity, emphasizing the importance of a dynamic view of normalization in neural coding.

Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term.

Making predictions about the rewards associated with environmental stimuli and updating those predictions through feedback is an essential aspect of adaptive behavior. Theorists have argued that dopamine encodes a reward prediction error (RPE) signal that is used in such a reinforcement learning process. Recent work with fMRI has demonstrated that the BOLD signal in dopaminergic target areas meets both necessary and sufficient conditions of an axiomatic model of the RPE hypothesis. However, there has been no direct evidence that dopamine release itself also meets necessary and sufficient criteria for encoding an RPE signal. Further, the fact that dopamine neurons have low tonic firing rates that yield a limited dynamic range for encoding negative RPEs has led to significant debate about whether positive and negative prediction errors are encoded on a similar scale. To address both of these issues, we used fast-scan cyclic voltammetry to measure reward-evoked dopamine release at carbon fiber electrodes chronically implanted in the nucleus accumbens core of rats trained on a probabilistic decision-making task. We demonstrate that dopamine concentrations transmit a bidirectional RPE signal with symmetrical encoding of positive and negative RPEs. Our findings strengthen the case that changes in dopamine concentration alone are sufficient to encode the full range of RPEs necessary for reinforcement learning.

Neuroanatomy predicts individual risk attitudes.

Over the course of the last decade a multitude of studies have investigated the relationship between neural activations and individual human decision-making. Here we asked whether the anatomical features of individual human brains could be used to predict the fundamental preferences of human choosers. To that end, we quantified the risk attitudes of human decision-makers using standard economic tools and quantified the gray matter cortical volume in all brain areas using standard neurobiological tools. Our whole-brain analysis revealed that the gray matter volume of a region in the right posterior parietal cortex was significantly predictive of individual risk attitudes. Participants with higher gray matter volume in this region exhibited less risk aversion. To test the robustness of this finding we examined a second group of participants and used econometric tools to test the ex ante hypothesis that gray matter volume in this area predicts individual risk attitudes. Our finding was confirmed in this second group. Our results, while being silent about causal relationships, identify what might be considered the first stable biomarker for financial risk-attitude. If these results, gathered in a population of midlife northeast American adults, hold in the general population, they will provide constraints on the possible neural mechanisms underlying risk attitudes. The results will also provide a simple measurement of risk attitudes that could be easily extracted from abundance of existing medical brain scans, and could potentially provide a characteristic distribution of these attitudes for policy makers.

Adolescents' risk-taking behavior is driven by tolerance to ambiguity.

Adolescents engage in a wide range of risky behaviors that their older peers shun, and at an enormous cost. Despite being older, stronger, and healthier than children, adolescents face twice the risk of mortality and morbidity faced by their younger peers. Are adolescents really risk-seekers or does some richer underlying preference drive their love of the uncertain? To answer that question, we used standard experimental economic methods to assess the attitudes of 65 individuals ranging in age from 12 to 50 toward risk and ambiguity. Perhaps surprisingly, we found that adolescents were, if anything, more averse to clearly stated risks than their older peers. What distinguished adolescents was their willingness to accept ambiguous conditions--situations in which the likelihood of winning and losing is unknown. Though adults find ambiguous monetary lotteries undesirable, adolescents find them tolerable. This finding suggests that the higher level of risk-taking observed among adolescents may reflect a higher tolerance for the unknown. Biologically, such a tolerance may make sense, because it would allow young organisms to take better advantage of learning opportunities; it also suggests that policies that seek to inform adolescents of the risks, costs, and benefits of unexperienced dangerous behaviors may be effective and, when appropriate, could be used to complement policies that limit their experiences.

Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis.

A number of recent advances have been achieved in the study of midbrain dopaminergic neurons. Understanding these advances and how they relate to one another requires a deep understanding of the computational models that serve as an explanatory framework and guide ongoing experimental inquiry. This intertwining of theory and experiment now suggests very clearly that the phasic activity of the midbrain dopamine neurons provides a global mechanism for synaptic modification. These synaptic modifications, in turn, provide the mechanistic underpinning for a specific class of reinforcement learning mechanisms that now seem to underlie much of human and animal behavior. This review describes both the critical empirical findings that are at the root of this conclusion and the fantastic theoretical advances from which this conclusion is drawn.

Origins of noise in both improving and degrading decision making.

Noise is a fundamental problem for information processing in neural systems. In decision-making, noise is assumed to have a primary role in errors and stochastic choice behavior. However, little is known about how noise arising from different sources contributes to value coding and choice behaviors, especially when it interacts with neural computation. Here we examine how noise arising early versus late in the choice process differentially impacts context-dependent choice behavior. We found in model simulations that early and late noise predict opposing context effects: under early noise, contextual information enhances choice accuracy; while under late noise, context degrades choice accuracy. Furthermore, we verified these opposing predictions in experimental human choice behavior. Manipulating early and late noise - by inducing uncertainty in option values and controlling time pressure - produced dissociable positive and negative context effects. These findings reconcile controversial experimental findings in the literature reporting either context-driven impairments or improvements in choice performance, suggesting a unified mechanism for context-dependent choice. More broadly, these findings highlight how different sources of noise can interact with neural computations to differentially modulate behavior. Significance: The current study addresses the role of noise origin in decision-making, reconciling controversies around how decision-making is impacted by context. We demonstrate that different types of noise - either arising early during evaluation or late during option comparison - leads to distinct results: with early noise, context enhances choice accuracy, while with late noise, context impairs it. Understanding these dynamics offers potential strategies for improving decision-making in noisy environments and refining existing neural computation models. Overall, our findings advance our understanding of how neural systems handle noise in essential cognitive tasks, suggest a beneficial role for contextual modulation under certain conditions, and highlight the profound implications of noise structure in decision-making.

Factors associated with loneliness, depression, and anxiety during the early stages of the COVID-19 pandemic.

The COVID-19 pandemic was an unparalleled stressor that enhanced isolation. Loneliness has been identified as an epidemic by the US Surgeon General. This study aimed to: (1) characterize longitudinal trajectories of loneliness during the acute phase of the COVID-19 pandemic; (2) identify longitudinal mediators of the relationship of loneliness with anxiety and depression; and (3) examine how loneliness naturally clusters and identify factors associated with high loneliness. Two hundred and twenty-nine adults (78% female; mean age = 39.5 ± 13.8) completed an abbreviated version of the UCLA Loneliness Scale, Perceived Stress Scale, Emotion Regulation Questionnaire, State Anxiety Inventory, and Patient Health Questionnaire-8 longitudinally between April 2020 and 2021. Trajectory analyses demonstrated relatively stable loneliness over time, while anxiety and depression symptoms declined. Longitudinal analyses indicated that loneliness effects on anxiety and depression were both partially mediated by perceived stress, while emotion regulation capacity only mediated effects on anxiety. Three stable clusters of loneliness trajectories emerged (high, moderate, and low). The odds of moderate or high loneliness cluster membership were positively associated with higher perceived stress and negatively associated with greater cognitive reappraisal use. Our results demonstrate the important interconnections between loneliness and facets of mental health throughout the early phases of the pandemic and may inform targeted future interventions for loneliness work.

Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex.

Evidence from monkeys and humans suggests that the orbitofrontal cortex (OFC) encodes the subjective value of options under consideration during choice. Data from non-human primates suggests that these value signals are context-dependent, representing subjective value in a way influenced by the decision makers' recent experience. Using electrodes distributed throughout cortical and subcortical structures, human epilepsy patients performed an auction task where they repeatedly reported the subjective values they placed on snack food items. High-gamma activity in many cortical and subcortical sites including the OFC positively correlated with subjective value. Other OFC sites showed signals contextually modulated by the subjective value of previously offered goods-a context dependency predicted by theory but not previously observed in humans. These results suggest that value and value-context signals are simultaneously present but separately represented in human frontal cortical activity.

The nematode worm C. elegans chooses between bacterial foods as if maximizing economic utility.

In value-based decision making, options are selected according to subjective values assigned by the individual to available goods and actions. Despite the importance of this faculty of the mind, the neural mechanisms of value assignments, and how choices are directed by them, remain obscure. To investigate this problem, we used a classic measure of utility maximization, the Generalized Axiom of Revealed Preference, to quantify internal consistency of food preferences in Caenorhabditis elegans, a nematode worm with a nervous system of only 302 neurons. Using a novel combination of microfluidics and electrophysiology, we found that C. elegans food choices fulfill the necessary and sufficient conditions for utility maximization, indicating that nematodes behave as if they maintain, and attempt to maximize, an underlying representation of subjective value. Food choices are well-fit by a utility function widely used to model human consumers. Moreover, as in many other animals, subjective values in C. elegans are learned, a process we find requires intact dopamine signaling. Differential responses of identified chemosensory neurons to foods with distinct growth potentials are amplified by prior consumption of these foods, suggesting that these neurons may be part of a value-assignment system. The demonstration of utility maximization in an organism with a very small nervous system sets a new lower bound on the computational requirements for utility maximization and offers the prospect of an essentially complete explanation of value-based decision making at single neuron resolution in this organism.

Comparing apples and oranges: using reward-specific and reward-general subjective value representation in the brain.

The ability of human subjects to choose between disparate kinds of rewards suggests that the neural circuits for valuing different reward types must converge. Economic theory suggests that these convergence points represent the subjective values (SVs) of different reward types on a common scale for comparison. To examine these hypotheses and to map the neural circuits for reward valuation we had food and water-deprived subjects make risky choices for money, food, and water both in and out of a brain scanner. We found that risk preferences across reward types were highly correlated; the level of risk aversion an individual showed when choosing among monetary lotteries predicted their risk aversion toward food and water. We also found that partially distinct neural networks represent the SVs of monetary and food rewards and that these distinct networks showed specific convergence points. The hypothalamic region mainly represented the SV for food, and the posterior cingulate cortex mainly represented the SV for money. In both the ventromedial prefrontal cortex (vmPFC) and striatum there was a common area representing the SV of both reward types, but only the vmPFC significantly represented the SVs of money and food on a common scale appropriate for choice in our data set. A correlation analysis demonstrated interactions across money and food valuation areas and the common areas in the vmPFC and striatum. This may suggest that partially distinct valuation networks for different reward types converge on a unified valuation network, which enables a direct comparison between different reward types and hence guides valuation and choice.

Reward value-based gain control: divisive normalization in parietal cortex.

The representation of value is a critical component of decision making. Rational choice theory assumes that options are assigned absolute values, independent of the value or existence of other alternatives. However, context-dependent choice behavior in both animals and humans violates this assumption, suggesting that biological decision processes rely on comparative evaluation. Here we show that neurons in the monkey lateral intraparietal cortex encode a relative form of saccadic value, explicitly dependent on the values of the other available alternatives. Analogous to extra-classical receptive field effects in visual cortex, this relative representation incorporates target values outside the response field and is observed in both stimulus-driven activity and baseline firing rates. This context-dependent modulation is precisely described by divisive normalization, indicating that this standard form of sensory gain control may be a general mechanism of cortical computation. Such normalization in decision circuits effectively implements an adaptive gain control for value coding and provides a possible mechanistic basis for behavioral context-dependent violations of rationality.

Choice from non-choice: predicting consumer preferences from blood oxygenation level-dependent signals obtained during passive viewing.

Decision-making is often viewed as a two-stage process, where subjective values are first assigned to each option and then the option of the highest value is selected. Converging evidence suggests that these subjective values are represented in the striatum and medial prefrontal cortex (MPFC). A separate line of evidence suggests that activation in the same areas represents the values of rewards even when choice is not required, as in classical conditioning tasks. However, it is unclear whether the same neural mechanism is engaged in both cases. To address this question we measured brain activation with functional magnetic resonance imaging while human subjects passively viewed individual consumer goods. We then sampled activation from predefined regions of interest and used it to predict subsequent choices between the same items made outside of the scanner. Our results show that activation in the striatum and MPFC in the absence of choice predicts subsequent choices, suggesting that these brain areas represent value in a similar manner whether or not choice is required.