Quantifying the subjective cost of self-control in humans.

Since Odysseus committed to resisting the Sirens, mechanisms to limit self-control failure have been a central feature of human behavior. Psychologists have long argued that the use of self-control is an effortful process and, more recently, that its failure arises when the cognitive costs of self-control outweigh its perceived benefits. In a similar way, economists have argued that sophisticated choosers can adopt "precommitment strategies" that tie the hands of their future selves in order to reduce these costs. Yet, we still lack an empirical tool to quantify and demonstrate the cost of self-control. Here, we develop and validate an economic decision-making task to quantify the subjective cost of self-control by determining the monetary cost a person is willing to incur in order to eliminate the need for self-control. We find that humans will pay to avoid having to exert self-control in a way that scales with increasing levels of temptation and that these costs appear to be modulated both by motivational incentives and stress exposure. Our psychophysical approach allows us to index moment-to-moment self-control costs at the within-subject level, validating important theoretical work across multiple disciplines and opening avenues of self-control research in healthy and clinical populations.

The computational form of craving is a selective multiplication of economic value.

Craving is thought to be a specific desire state that biases choice toward the desired object, be it chocolate or drugs. A vast majority of people report having experienced craving of some kind. In its pathological form craving contributes to health outcomes in addiction and obesity. Yet despite its ubiquity and clinical relevance we still lack a basic neurocomputational understanding of craving. Here, using an instantaneous measure of subjective valuation and selective cue exposure, we identify a behavioral signature of a food craving-like state and advance a computational framework for understanding how this state might transform valuation to bias choice. We find desire induced by exposure to a specific high-calorie, high-fat/sugar snack good is expressed in subjects' momentary willingness to pay for this good. This effect is selective but not exclusive to the exposed good; rather, we find it generalizes to nonexposed goods in proportion to their subjective attribute similarity to the exposed ones. A second manipulation of reward size (number of snack units available for purchase) further suggested that a multiplicative gain mechanism supports the transformation of valuation during laboratory craving. These findings help explain how real-world food craving can result in behaviors inconsistent with preferences expressed in the absence of craving and open a path for the computational modeling of craving-like phenomena using a simple and repeatable experimental tool for assessing subjective states in economic terms.

Normalized value coding explains dynamic adaptation in the human valuation process.

The notion of subjective value is central to choice theories in ecology, economics, and psychology, serving as an integrated decision variable by which options are compared. Subjective value is often assumed to be an absolute quantity, determined in a static manner by the properties of an individual option. Recent neurobiological studies, however, have shown that neural value coding dynamically adapts to the statistics of the recent reward environment, introducing an intrinsic temporal context dependence into the neural representation of value. Whether valuation exhibits this kind of dynamic adaptation at the behavioral level is unknown. Here, we show that the valuation process in human subjects adapts to the history of previous values, with current valuations varying inversely with the average value of recently observed items. The dynamics of this adaptive valuation are captured by divisive normalization, linking these temporal context effects to spatial context effects in decision making as well as spatial and temporal context effects in perception. These findings suggest that adaptation is a universal feature of neural information processing and offer a unifying explanation for contextual phenomena in fields ranging from visual psychophysics to economic choice.

The Functional Roles of the Amygdala and Prefrontal Cortex in Processing Uncertainty.

Decisions under uncertainty distinguish between those made under risk (known probabilities) and those made under ambiguity (unknown probabilities). Despite widespread interest in decisions under uncertainty and the successful documentation that these distinct psychological constructs profoundly-and differentially-impact behavior, research has not been able to systematically converge on which brain regions are functionally involved in processing risk and ambiguity. We merge a lesion approach with computational modeling and simultaneous measurement of the arousal response to investigate the impact the medial prefrontal cortex (mPFC), lateral prefrontal cortex (lPFC), and amygdala have on decisions under uncertainty. Results reveal that the lPFC acts as a unitary system for processing uncertainty: Lesions to this region disrupted the relationship between arousal and choice, broadly increasing both risk and ambiguity seeking. In contrast, the mPFC and amygdala appeared to play no role in processing risk, and the mPFC only had a tenuous relationship with ambiguous uncertainty. Together, these findings reveal that only the lPFC plays a global role in processing the highly aversive nature of uncertainty.

Choice-theoretic foundations of the divisive normalization model.

Recent advances in neuroscience suggest that a utility-like calculation is involved in how the brain makes choices, and that this calculation may use a computation known as divisive normalization. While this tells us how the brain makes choices, it is not immediately evident why the brain uses this computation or exactly what behavior is consistent with it. In this paper, we address both of these questions by proving a three-way equivalence theorem between the normalization model, an information-processing model, and an axiomatic characterization. The information-processing model views behavior as optimally balancing the expected value of the chosen object against the entropic cost of reducing stochasticity in choice. This provides an optimality rationale for why the brain may have evolved to use normalization-type models. The axiomatic characterization gives a set of testable behavioral statements equivalent to the normalization model. This answers what behavior arises from normalization. Our equivalence result unifies these three models into a single theory that answers the "how", "why", and "what" of choice behavior.

The Reduction of Ventrolateral Prefrontal Cortex Gray Matter Volume Correlates with Loss of Economic Rationality in Aging.

The population of people above 65 years old continues to grow, and there is mounting evidence that as humans age they are more likely to make errors. However, the specific effect of neuroanatomical aging on the efficiency of economic decision-making is poorly understood. We used whole-brain voxel-based morphometry analysis to determine where reduction of gray matter volume in healthy female and male adults over the age of 65 years correlates with a classic measure of economic irrationality: violations of the Generalized Axiom of Revealed Preference. All participants were functionally normal with Mini-Mental State Examination scores ranging between 26 and 30. While our elders showed the previously reported decline in rationality compared with younger subjects, chronological age per se did not correlate with rationality measures within our population of elders. Instead, reduction of gray matter density in ventrolateral prefrontal cortex correlates tightly with irrational behavior. Interestingly, using a large fMRI sample and meta-analytic tool with Neurosynth, we found that this brain area shows strong coactivation patterns with nearly all of the value-associated regions identified in previous studies. These findings point toward a neuroanatomic locus for economic rationality in the aging brain and highlight the importance of understanding both anatomy and function in the study of aging, cognition, and decision-making.SIGNIFICANCE STATEMENT Age is a crucial factor in decision-making, with older individuals making more errors in choices. Using whole-brain voxel-based morphometry analysis, we found that reduction of gray matter density in ventrolateral prefrontal cortex correlates with economic irrationality: reduced gray matter volume in this area correlates with the frequency and severity of violations of the Generalized Axiom of Revealed Preference. Furthermore, this brain area strongly coactivates with other reward-associated regions identified with Neurosynth. These findings point toward a role for neuroscientific discoveries in shaping long-standing economic views of decision-making.

Impairment of decision-making in multiple sclerosis: A neuroeconomic approach.

OBJECTIVE: To assess the decision-making impairment in patients with multiple sclerosis (MS) and how they relate to other cognitive domains. METHODS: We performed a cross-sectional analysis in 84 patients with MS, and 21 matched healthy controls using four tasks taken from behavioral economics: (1) risk preferences, (2) choice consistency, (3) delay of gratification, and (4) rate of learning. All tasks were conducted using real-world reward outcomes (food or money) in different real-life conditions. Participants underwent cognitive examination using the Brief Repeatable Battery-Neuropsychology. RESULTS: Patients showed higher risk aversion (general propensity to choose the lottery was 0.51 vs 0.64, p = 0.009), a trend to choose more immediate rewards over larger but delayed rewards ( p = 0.108), and had longer reactions times ( p = 0.033). Choice consistency and learning rates were not different between groups. Progressive patients chose slower than relapsing patients. In relation to general cognitive impairments, we found correlations between impaired decision-making and impaired verbal memory ( r = 0.29, p = 0.009), visual memory ( r = -0.37, p = 0.001), and reduced processing speed ( r = -0.32, p = 0.001). Normalized gray matter volume correlated with deliberation time ( r = -0.32, p = 0.005). CONCLUSION: Patients with MS suffer significant decision-making impairments, even at the early stages of the disease, and may affect patients' quality and social life.

Modulation of Saccade Vigor during Value-Based Decision Making.

During value-based decision-making, individuals consider the various options and select the one that provides the maximum subjective value. Although the brain integrates abstract information to compute and compare these values, the only behavioral outcome is often the decision itself. However, if the options are visual stimuli, during deliberation the brain moves the eyes from one stimulus to the other. Previous work suggests that saccade vigor, i.e., peak velocity as a function of amplitude, is greater if reward is associated with the visual stimulus. This raises the possibility that vigor during the free viewing of options may be influenced by the valuation of each option. Here, humans chose between a small, immediate monetary reward and a larger but delayed reward. As the deliberation began, vigor was similar for the saccades made to the two options but diverged 0.5 s before decision time, becoming greater for the preferred option. This difference in vigor increased as a function of the difference in the subjective values that the participant assigned to the delayed and immediate options. After the decision was made, participants continued to gaze at the options, but with reduced vigor, making it possible to infer timing of the decision from the sudden drop in vigor. Therefore, the subjective value that the brain assigned to a stimulus during decision-making affected the motor system via the vigor with which the eyes moved toward that stimulus. SIGNIFICANCE STATEMENT: We find that, as individuals deliberate between two rewarding options and arrive at a decision, the vigor with which they make saccades to each option reflects a real-time evaluation of that option. With deliberation, saccade vigor diverges between the two options, becoming greater for the option that the individual will eventually choose. The results suggest a shared element between the network that assigns value to a stimulus during the process of decision-making and the network that controls vigor of movements toward that stimulus.

Unexpected but Incidental Positive Outcomes Predict Real-World Gambling.

Positive mood can affect a person's tendency to gamble, possibly because positive mood fosters unrealistic optimism. At the same time, unexpected positive outcomes, often called prediction errors, influence mood. However, a linkage between positive prediction errors-the difference between expected and obtained outcomes-and consequent risk taking has yet to be demonstrated. Using a large data set of New York City lottery gambling and a model inspired by computational accounts of reward learning, we found that people gamble more when incidental outcomes in the environment (e.g., local sporting events and sunshine) are better than expected. When local sports teams performed better than expected, or a sunny day followed a streak of cloudy days, residents gambled more. The observed relationship between prediction errors and gambling was ubiquitous across the city's socioeconomically diverse neighborhoods and was specific to sports and weather events occurring locally in New York City. Our results suggest that unexpected but incidental positive outcomes influence risk taking.

Normalization is a general neural mechanism for context-dependent decision making.

Understanding the neural code is critical to linking brain and behavior. In sensory systems, divisive normalization seems to be a canonical neural computation, observed in areas ranging from retina to cortex and mediating processes including contrast adaptation, surround suppression, visual attention, and multisensory integration. Recent electrophysiological studies have extended these insights beyond the sensory domain, demonstrating an analogous algorithm for the value signals that guide decision making, but the effects of normalization on choice behavior are unknown. Here, we show that choice models using normalization generate significant (and classically irrational) choice phenomena driven by either the value or number of alternative options. In value-guided choice experiments, both monkey and human choosers show novel context-dependent behavior consistent with normalization. These findings suggest that the neural mechanism of value coding critically influences stochastic choice behavior and provide a generalizable quantitative framework for examining context effects in decision making.

Thirst-dependent risk preferences in monkeys identify a primitive form of wealth.

Experimental economic techniques have been widely used to evaluate human risk attitudes, but how these measured attitudes relate to overall individual wealth levels is unclear. Previous noneconomic work has addressed this uncertainty in animals by asking the following: (i) Do our close evolutionary relatives share both our risk attitudes and our degree of economic rationality? And (ii) how does the amount of food or water one holds (a nonpecuniary form of "wealth") alter risk attitudes in these choosers? Unfortunately, existing noneconomic studies have provided conflicting insights from an economic point of view. We therefore used standard techniques from human experimental economics to measure monkey risk attitudes for water rewards as a function of blood osmolality (an objective measure of how much water the subjects possess). Early in training, monkeys behaved randomly, consistently violating first-order stochastic dominance and monotonicity. After training, they behaved like human choosers--technically consistent in their choices and weakly risk averse (i.e., risk averse or risk neutral on average)--suggesting that well-trained monkeys can serve as a model for human choice behavior. As with attitudes about money in humans, these risk attitudes were strongly wealth dependent; as the animals became "poorer," risk aversion increased, a finding incompatible with some models of wealth and risk in human decision making.

Like cognitive function, decision making across the life span shows profound age-related changes.

It has long been known that human cognitive function improves through young adulthood and then declines across the later life span. Here we examined how decision-making function changes across the life span by measuring risk and ambiguity attitudes in the gain and loss domains, as well as choice consistency, in an urban cohort ranging in age from 12 to 90 y. We identified several important age-related patterns in decision making under uncertainty: First, we found that healthy elders between the ages of 65 and 90 were strikingly inconsistent in their choices compared with younger subjects. Just as elders show profound declines in cognitive function, they also show profound declines in choice rationality compared with their younger peers. Second, we found that the widely documented phenomenon of ambiguity aversion is specific to the gain domain and does not occur in the loss domain, except for a slight effect in older adults. Finally, extending an earlier report by our group, we found that risk attitudes across the life span show an inverted U-shaped function; both elders and adolescents are more risk-averse than their midlife counterparts. Taken together, these characterizations of decision-making function across the life span in this urban cohort strengthen the conclusions of previous reports suggesting a profound impact of aging on cognitive function in this domain.

Dynamic divisive normalization predicts time-varying value coding in decision-related circuits.

Normalization is a widespread neural computation, mediating divisive gain control in sensory processing and implementing a context-dependent value code in decision-related frontal and parietal cortices. Although decision-making is a dynamic process with complex temporal characteristics, most models of normalization are time-independent and little is known about the dynamic interaction of normalization and choice. Here, we show that a simple differential equation model of normalization explains the characteristic phasic-sustained pattern of cortical decision activity and predicts specific normalization dynamics: value coding during initial transients, time-varying value modulation, and delayed onset of contextual information. Empirically, we observe these predicted dynamics in saccade-related neurons in monkey lateral intraparietal cortex. Furthermore, such models naturally incorporate a time-weighted average of past activity, implementing an intrinsic reference-dependence in value coding. These results suggest that a single network mechanism can explain both transient and sustained decision activity, emphasizing the importance of a dynamic view of normalization in neural coding.

Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term.

Making predictions about the rewards associated with environmental stimuli and updating those predictions through feedback is an essential aspect of adaptive behavior. Theorists have argued that dopamine encodes a reward prediction error (RPE) signal that is used in such a reinforcement learning process. Recent work with fMRI has demonstrated that the BOLD signal in dopaminergic target areas meets both necessary and sufficient conditions of an axiomatic model of the RPE hypothesis. However, there has been no direct evidence that dopamine release itself also meets necessary and sufficient criteria for encoding an RPE signal. Further, the fact that dopamine neurons have low tonic firing rates that yield a limited dynamic range for encoding negative RPEs has led to significant debate about whether positive and negative prediction errors are encoded on a similar scale. To address both of these issues, we used fast-scan cyclic voltammetry to measure reward-evoked dopamine release at carbon fiber electrodes chronically implanted in the nucleus accumbens core of rats trained on a probabilistic decision-making task. We demonstrate that dopamine concentrations transmit a bidirectional RPE signal with symmetrical encoding of positive and negative RPEs. Our findings strengthen the case that changes in dopamine concentration alone are sufficient to encode the full range of RPEs necessary for reinforcement learning.

Neuroanatomy predicts individual risk attitudes.

Over the course of the last decade a multitude of studies have investigated the relationship between neural activations and individual human decision-making. Here we asked whether the anatomical features of individual human brains could be used to predict the fundamental preferences of human choosers. To that end, we quantified the risk attitudes of human decision-makers using standard economic tools and quantified the gray matter cortical volume in all brain areas using standard neurobiological tools. Our whole-brain analysis revealed that the gray matter volume of a region in the right posterior parietal cortex was significantly predictive of individual risk attitudes. Participants with higher gray matter volume in this region exhibited less risk aversion. To test the robustness of this finding we examined a second group of participants and used econometric tools to test the ex ante hypothesis that gray matter volume in this area predicts individual risk attitudes. Our finding was confirmed in this second group. Our results, while being silent about causal relationships, identify what might be considered the first stable biomarker for financial risk-attitude. If these results, gathered in a population of midlife northeast American adults, hold in the general population, they will provide constraints on the possible neural mechanisms underlying risk attitudes. The results will also provide a simple measurement of risk attitudes that could be easily extracted from abundance of existing medical brain scans, and could potentially provide a characteristic distribution of these attitudes for policy makers.

Adolescents' risk-taking behavior is driven by tolerance to ambiguity.

Adolescents engage in a wide range of risky behaviors that their older peers shun, and at an enormous cost. Despite being older, stronger, and healthier than children, adolescents face twice the risk of mortality and morbidity faced by their younger peers. Are adolescents really risk-seekers or does some richer underlying preference drive their love of the uncertain? To answer that question, we used standard experimental economic methods to assess the attitudes of 65 individuals ranging in age from 12 to 50 toward risk and ambiguity. Perhaps surprisingly, we found that adolescents were, if anything, more averse to clearly stated risks than their older peers. What distinguished adolescents was their willingness to accept ambiguous conditions--situations in which the likelihood of winning and losing is unknown. Though adults find ambiguous monetary lotteries undesirable, adolescents find them tolerable. This finding suggests that the higher level of risk-taking observed among adolescents may reflect a higher tolerance for the unknown. Biologically, such a tolerance may make sense, because it would allow young organisms to take better advantage of learning opportunities; it also suggests that policies that seek to inform adolescents of the risks, costs, and benefits of unexperienced dangerous behaviors may be effective and, when appropriate, could be used to complement policies that limit their experiences.

Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis.

A number of recent advances have been achieved in the study of midbrain dopaminergic neurons. Understanding these advances and how they relate to one another requires a deep understanding of the computational models that serve as an explanatory framework and guide ongoing experimental inquiry. This intertwining of theory and experiment now suggests very clearly that the phasic activity of the midbrain dopamine neurons provides a global mechanism for synaptic modification. These synaptic modifications, in turn, provide the mechanistic underpinning for a specific class of reinforcement learning mechanisms that now seem to underlie much of human and animal behavior. This review describes both the critical empirical findings that are at the root of this conclusion and the fantastic theoretical advances from which this conclusion is drawn.

Synchronized Incidental Affect Changes Ambiguity Preferences.

Decisions under uncertainty are prevalent, but come under two distinct types. Risk, which has unknown outcomes but known probabilities for those outcomes and ambiguity which contains both unknown outcomes and unknown probabilities. Although there have been several studies linking affect and aversion to ambiguity, there have been no studies that have to identify how changing one's affective response can change their choices. A total of 166 adults ( M = 36.54, SD = 11.80) participated in an online study through Prolific. Participants were presented with a lottery on each trial which varied on its uncertainty type (risky vs ambiguous) and winning characteristics (winning probability and amount). Half of the ambiguous lotteries were paired with an neutral image (e.g., office supplies), while the other half was paired with an emotionally evocative image (e.g., burning house) that was hypothesized to incidentally influence their decisions. As measured by both raw choice data as well as through a computational model, participants were more averse to ambiguity when the lottery was paired with an emotionally evocative image. Follow-up analyses revealed that only lotteries in which the computational model predicted the participant would choose the lottery were affected by the images. This study highlights the phenomenon in which one's awareness of an affective stimulus can alter its impact on their decisions.

Origins of noise in both improving and degrading decision making.

Noise is a fundamental problem for information processing in neural systems. In decision-making, noise is assumed to have a primary role in errors and stochastic choice behavior. However, little is known about how noise arising from different sources contributes to value coding and choice behaviors, especially when it interacts with neural computation. Here we examine how noise arising early versus late in the choice process differentially impacts context-dependent choice behavior. We found in model simulations that early and late noise predict opposing context effects: under early noise, contextual information enhances choice accuracy; while under late noise, context degrades choice accuracy. Furthermore, we verified these opposing predictions in experimental human choice behavior. Manipulating early and late noise - by inducing uncertainty in option values and controlling time pressure - produced dissociable positive and negative context effects. These findings reconcile controversial experimental findings in the literature reporting either context-driven impairments or improvements in choice performance, suggesting a unified mechanism for context-dependent choice. More broadly, these findings highlight how different sources of noise can interact with neural computations to differentially modulate behavior. Significance: The current study addresses the role of noise origin in decision-making, reconciling controversies around how decision-making is impacted by context. We demonstrate that different types of noise - either arising early during evaluation or late during option comparison - leads to distinct results: with early noise, context enhances choice accuracy, while with late noise, context impairs it. Understanding these dynamics offers potential strategies for improving decision-making in noisy environments and refining existing neural computation models. Overall, our findings advance our understanding of how neural systems handle noise in essential cognitive tasks, suggest a beneficial role for contextual modulation under certain conditions, and highlight the profound implications of noise structure in decision-making.

Factors associated with loneliness, depression, and anxiety during the early stages of the COVID-19 pandemic.

The COVID-19 pandemic was an unparalleled stressor that enhanced isolation. Loneliness has been identified as an epidemic by the US Surgeon General. This study aimed to: (1) characterize longitudinal trajectories of loneliness during the acute phase of the COVID-19 pandemic; (2) identify longitudinal mediators of the relationship of loneliness with anxiety and depression; and (3) examine how loneliness naturally clusters and identify factors associated with high loneliness. Two hundred and twenty-nine adults (78% female; mean age = 39.5 ± 13.8) completed an abbreviated version of the UCLA Loneliness Scale, Perceived Stress Scale, Emotion Regulation Questionnaire, State Anxiety Inventory, and Patient Health Questionnaire-8 longitudinally between April 2020 and 2021. Trajectory analyses demonstrated relatively stable loneliness over time, while anxiety and depression symptoms declined. Longitudinal analyses indicated that loneliness effects on anxiety and depression were both partially mediated by perceived stress, while emotion regulation capacity only mediated effects on anxiety. Three stable clusters of loneliness trajectories emerged (high, moderate, and low). The odds of moderate or high loneliness cluster membership were positively associated with higher perceived stress and negatively associated with greater cognitive reappraisal use. Our results demonstrate the important interconnections between loneliness and facets of mental health throughout the early phases of the pandemic and may inform targeted future interventions for loneliness work.