Fins, feathers, fingers, and finding an explanation for the puzzle of ephemeral rewards.

This article discusses the ephemeral reward task and how it is not always a clear and concise choice. This is demonstrated through some animal studies involving birds and primates. This article also shows that when compared to human studies, that there are positive correlations between the BART and optimal choice in the ephemeral reward task, meaning that those who took more risks also were more inclined to be optimal. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Using recurrent neural network to estimate irreducible stochasticity in human choice behavior.

Theoretical computational models are widely used to describe latent cognitive processes. However, these models do not equally explain data across participants, with some individuals showing a bigger predictive gap than others. In the current study, we examined the use of theory-independent models, specifically recurrent neural networks (RNNs), to classify the source of a predictive gap in the observed data of a single individual. This approach aims to identify whether the low predictability of behavioral data is mainly due to noisy decision-making or misspecification of the theoretical model. First, we used computer simulation in the context of reinforcement learning to demonstrate that RNNs can be used to identify model misspecification in simulated agents with varying degrees of behavioral noise. Specifically, both prediction performance and the number of RNN training epochs (i.e., the point of early stopping) can be used to estimate the amount of stochasticity in the data. Second, we applied our approach to an empirical dataset where the actions of low IQ participants, compared with high IQ participants, showed lower predictability by a well-known theoretical model (i.e., Daw's hybrid model for the two-step task). Both the predictive gap and the point of early stopping of the RNN suggested that model misspecification is similar across individuals. This led us to a provisional conclusion that low IQ subjects are mostly noisier compared to their high IQ peers, rather than being more misspecified by the theoretical model. We discuss the implications and limitations of this approach, considering the growing literature in both theoretical and data-driven computational modeling in decision-making science.

Plaudits for logits in sensory neuroscience.

A workhorse tool of economic decision-making has long sought to get inside people's heads through careful examination of their choices. In this issue of Neuron, Carandini1 flips the script, showing how it can model how the brain makes sensory choices.

A cognitive approach to better understand foraging strategies of the adult domestic hen.

Foraging is known to be one of the most important activities in the behavioral budget of chickens. However, how these animals adapt different foraging strategies to diverse environmental variations is currently poorly understood. To gain further insight into this matter, in the present study, hens were submitted to the sloped-tubes task. In this task, the experimenter can manipulate the information that enables the hens to find a food reward (visible or not), placed in one of two hollow tubes. First, 12 hens were tested under free-choice conditions (no penalty for exhaustive searching in both tubes). Under these conditions, the hens adopted a non-random, side-biased strategy when the food location was not directly visible. Then, we divided the hens in two cohorts of equal size to study deeper the hens' foraging strategy when faced (1) with a different container, or (2) with a restrictive environmental constraint under forced-choice conditions (no food reward if the unbaited tube is visited first). This latter constraint increased the risk of the hen not receiving food. A change in the containers didn't modify the search behavior of the hens. However, in forced-choice conditions when the location of the food was not directly visible, four out of six hens learned to choose by exclusion. We conclude that hens can selectively adapt their foraging strategy to the point of adopting an exclusion performance, depending on available information and environmental constraints (high or low risk).

Ground-nesting birds learn egg appearance to guide background choice for camouflage.

Camouflage is vital for the survival of many prey species1,2, including ground-nesting birds3,4,5,6. Egg camouflage via background matching and disruptive coloration (high contrast markings that break up the body outline) is often behaviourally mediated by selecting substrates that enhance egg camouflage1,2,3,4,5,6. However, the mechanisms controlling this behaviour in birds have remained unknown. Several, not mutually exclusive, mechanisms have been suggested to control background choice for egg camouflage7. These include where individual background preferences are genetically linked to egg coloration, enabled through learning egg appearances from previous breeding attempts, or modified by imprinting on visual backgrounds during early life7, Here, using predator vision models, we compared the camouflage of Japanese quail (Coturnix japonica) eggs among females who were allowed to choose one of four coloured substrates on which to lay3. Birds were divided into experienced females who had been given the opportunity to observe the appearance of their eggs, versus naïve females breeding for the first time. Our experiment revealed that breeding experience leads to improved background choices made for egg background matching. However, substrate choice for disruptive coloration appeared genetically determined, as both bird groups chose backgrounds that enhanced egg disruptiveness regardless of experience. These mechanisms underpin behaviours that are likely essential for birds and other animals to optimise camouflage and avoid predation6.

Direct stimulation of anterior insula and ventromedial prefrontal cortex disrupts economic choices.

Neural activity within the ventromedial prefrontal cortex (vmPFC) and anterior insula (aIns) is often associated with economic choices and confidence. However, it remains unclear whether these brain regions are causally related to these processes. To address this issue, we leveraged intracranial electrical stimulation (iES) data obtained from patients with epilepsy performing an economic choice task. Our results reveal opposite effects of stimulation on decision-making depending on its location along a dorso-ventral axis within each region. Specifically, stimulation of the ventral subregion within aIns reduces risk-taking by increasing participants' sensitivity to potential losses, whereas stimulation of the dorsal subregion of aIns and the ventral portion of the vmPFC increases risk-taking by reducing participants' sensitivity to losses. Moreover, stimulation of the aIns consistently decreases participants' confidence, regardless of its location within the aIns. These findings suggest the existence of functionally dissociated neural subregions and circuits causally involved in accepting or avoiding challenges.

Distinct roles of monkey OFC-subcortical pathways in adaptive behavior.

Primates must adapt to changing environments by optimizing their behavior to make beneficial choices. At the core of adaptive behavior is the orbitofrontal cortex (OFC) of the brain, which updates choice value through direct experience or knowledge-based inference. Here, we identify distinct neural circuitry underlying these two separate abilities. We designed two behavioral tasks in which two male macaque monkeys updated the values of certain items, either by directly experiencing changes in stimulus-reward associations, or by inferring the value of unexperienced items based on the task's rules. Chemogenetic silencing of bilateral OFC combined with mathematical model-fitting analysis revealed that monkey OFC is involved in updating item value based on both experience and inference. In vivo imaging of chemogenetic receptors by positron emission tomography allowed us to map projections from the OFC to the rostromedial caudate nucleus (rmCD) and the medial part of the mediodorsal thalamus (MDm). Chemogenetic silencing of the OFC-rmCD pathway impaired experience-based value updating, while silencing the OFC-MDm pathway impaired inference-based value updating. Our results thus demonstrate dissociable contributions of distinct OFC projections to different behavioral strategies, and provide new insights into the neural basis of value-based adaptive decision-making in primates.

Differential stability of task variable representations in retrosplenial cortex.

Cortical neurons store information across different timescales, from seconds to years. Although information stability is variable across regions, it can vary within a region as well. Association areas are known to multiplex behaviorally relevant variables, but the stability of their representations is not well understood. Here, we longitudinally recorded the activity of neuronal populations in the mouse retrosplenial cortex (RSC) during the performance of a context-choice association task. We found that the activity of neurons exhibits different levels of stability across days. Using linear classifiers, we quantified the stability of three task-relevant variables. We find that RSC representations of context and trial outcome display higher stability than motor choice, both at the single cell and population levels. Together, our findings show an important characteristic of association areas, where diverse streams of information are stored with varying levels of stability, which may balance representational reliability and flexibility according to behavioral demands.

Temporal regularities shape perceptual decisions and striatal dopamine signals.

Perceptual decisions should depend on sensory evidence. However, such decisions are also influenced by past choices and outcomes. These choice history biases may reflect advantageous strategies to exploit temporal regularities of natural environments. However, it is unclear whether and how observers can adapt their choice history biases to different temporal regularities, to exploit the multitude of temporal correlations that exist in nature. Here, we show that male mice adapt their perceptual choice history biases to different temporal regularities of visual stimuli. This adaptation was slow, evolving over hundreds of trials across several days. It occurred alongside a fast non-adaptive choice history bias, limited to a few trials. Both fast and slow trial history effects are well captured by a normative reinforcement learning algorithm with multi-trial belief states, comprising both current trial sensory and previous trial memory states. We demonstrate that dorsal striatal dopamine tracks predictions of the model and behavior, suggesting that striatal dopamine reports reward predictions associated with adaptive choice history biases. Our results reveal the adaptive nature of perceptual choice history biases and shed light on their underlying computational principles and neural correlates.

Influence of surprise on reinforcement learning in younger and older adults.

Surprise is a key component of many learning experiences, and yet its precise computational role, and how it changes with age, remain debated. One major challenge is that surprise often occurs jointly with other variables, such as uncertainty and outcome probability. To assess how humans learn from surprising events, and whether aging affects this process, we studied choices while participants learned from bandits with either Gaussian or bi-modal outcome distributions, which decoupled outcome probability, uncertainty, and surprise. A total of 102 participants (51 older, aged 50-73; 51 younger, 19-30 years) chose between three bandits, one of which had a bimodal outcome distribution. Behavioral analyses showed that both age-groups learned the average of the bimodal bandit less well. A trial-by-trial analysis indicated that participants performed choice reversals immediately following large absolute prediction errors, consistent with heightened sensitivity to surprise. This effect was stronger in older adults. Computational models indicated that learning rates in younger as well as older adults were influenced by surprise, rather than uncertainty, but also suggested large interindividual variability in the process underlying learning in our task. Our work bridges between behavioral economics research that has focused on how outcomes with low probability affect choice in older adults, and reinforcement learning work that has investigated age differences in the effects of uncertainty and suggests that older adults overly adapt to surprising events, even when accounting for probability and uncertainty effects.

Beyond discrete-choice options.

While decision theories have evolved over the past five decades, their focus has largely been on choices among a limited number of discrete options, even though many real-world situations have a continuous-option space. Recently, theories have attempted to address decisions with continuous-option spaces, and several computational models have been proposed within the sequential sampling framework to explain how we make a decision in continuous-option space. This article aims to review the main attempts to understand decisions on continuous-option spaces, give an overview of applications of these types of decisions, and present puzzles to be addressed by future developments.

Cracking and Packing Information about the Features of Expected Rewards in the Orbitofrontal Cortex.

The orbitofrontal cortex (OFC) is crucial for tracking various aspects of expected outcomes, thereby helping to guide choices and support learning. Our previous study showed that the effects of reward timing and size on the activity of single units in OFC were dissociable when these attributes were manipulated independently ( Roesch et al., 2006). However, in real-life decision-making scenarios, outcome features often change simultaneously, so here we investigated how OFC neurons in male rats integrate information about the timing and identity (flavor) of reward and respond to changes in these features, according to whether they were changed simultaneously or separately. We found that a substantial number of OFC neurons fired differentially to immediate versus delayed reward and to the different reward flavors. However, contrary to the previous study, selectivity for timing was strongly correlated with selectivity for identity. Taken together with the previous research, these results suggest that when reward features are correlated, OFC tends to "pack" them into unitary constructs, whereas when they are independent, OFC tends to "crack" them into separate constructs. Furthermore, we found that when both reward timing and flavor were changed, reward-responsive OFC neurons showed unique activity patterns preceding and during the omission of an expected reward. Interestingly, this OFC activity is similar and slightly preceded the ventral tegmental area dopamine (VTA DA) activity observed in a previous study ( Takahashi et al., 2023), consistent with the role of OFC in providing predictive information to VTA DA neurons.

Decision-Making with Predictions of Others' Likely and Unlikely Choices in the Human Brain.

For better decisions in social interactions, humans often must understand the thinking of others and predict their actions. Since such predictions are uncertain, multiple predictions may be necessary for better decision-making. However, the neural processes and computations underlying such social decision-making remain unclear. We investigated this issue by developing a behavioral paradigm and performing functional magnetic resonance imaging and computational modeling. In our task, female and male participants were required to predict others' choices in order to make their own value-based decisions, as the outcome depended on others' choices. Results showed, to make choices, the participants mostly relied on a value difference (primary) generated from the case where others would make a likely choice, but sometimes they additionally used another value difference (secondary) from the opposite case where others make an unlikely choice. We found that the activations in the posterior cingulate cortex (PCC) correlated with the primary difference while the activations in the right dorsolateral prefrontal cortex (rdlPFC) correlated with the secondary difference. Analysis of neural coupling and temporal dynamics suggested a three-step processing network, beginning with the left amygdala signals for predictions of others' choices. Modulated by these signals, the PCC and rdlPFC reflect the respective value differences for self-decisions. Finally, the medial prefrontal cortex integrated these decision signals for a final decision. Our findings elucidate the neural process of constructing value-based decisions by predicting others and illuminate their key variables with social modulations, providing insight into the differential functional roles of these brain regions in this process.

The consequences of AI training on human decision-making.

AI is now an integral part of everyday decision-making, assisting us in both routine and high-stakes choices. These AI models often learn from human behavior, assuming this training data is unbiased. However, we report five studies that show that people change their behavior to instill desired routines into AI, indicating this assumption is invalid. To show this behavioral shift, we recruited participants to play the ultimatum game, where they were asked to decide whether to accept proposals of monetary splits made by either other human participants or AI. Some participants were informed their choices would be used to train an AI proposer, while others did not receive this information. Across five experiments, we found that people modified their behavior to train AI to make fair proposals, regardless of whether they could directly benefit from the AI training. After completing this task once, participants were invited to complete this task again but were told their responses would not be used for AI training. People who had previously trained AI persisted with this behavioral shift, indicating that the new behavioral routine had become habitual. This work demonstrates that using human behavior as training data has more consequences than previously thought since it can engender AI to perpetuate human biases and cause people to form habits that deviate from how they would normally act. Therefore, this work underscores a problem for AI algorithms that aim to learn unbiased representations of human preferences.

Task-specific relationships between error-related ERPs and behavior: Flanker, Stroop, and Go/Nogo tasks.

Performance monitoring has been widely studied during different forced-choice response tasks. Participants typically show longer response times (RTs) and increased accuracy following errors, but there are inconsistencies regarding the connection between error-related event-related brain potentials (ERPs) and behavior, such as RT and accuracy. The specific task in any given study could contribute to these inconsistencies, as different tasks may require distinct cognitive processes that impact ERP-behavior relationships. The present study sought to determine whether task moderates ERP-behavior relationships and whether these relationships are robustly observed when tasks and stimuli are treated as random effects. ERPs and behavioral indices (RTs and accuracy) recorded during flanker, Stroop, and Go/Nogo tasks from 180 people demonstrated a task-specific effect on ERP-behavior relationships, such that larger previous-trial error-related negativity (ERN) predicted longer RTs and greater likelihood of a correct response on subsequent trials during flanker and Stroop tasks but not during Go/Nogo task. Additionally, larger previous-trial error positivity (Pe) predicted faster RTs and smaller variances of RTs on subsequent trials for Stroop and Go/Nogo tasks but not for flanker task. When tasks and stimuli were treated as random effects, ERP-behavior relationships were not observed. These findings support the need to consider the task used for recording performance monitoring measures when interpreting results across studies.

Exploring the neuropsychological basis of behavioral contagion during learning about another agent's social preferences: Evidence from an ERP study.

Social contagion is a pervasive phenomenon and an important social influence that involves the rapid dissemination (propagation) of behaviors, attitudes, emotions, or ideas from one person to another, often without conscious reflection or rational thought. This phenomenon is closely related to conformity, by which a person changes his/her original ideas and attitude and imitates certain behavior of others. Although some behavioral research has been carried out on contagion and conformity, there is very little neuropsychological understanding of these phenomena. Existing research on social influence and conformity has predominantly focused on tasks like mental rotation or rating tasks involving facial expressions, with fewer studies exploring risk preferences and temporal discounting. However, there is a notable gap in the literature when it comes to examining social influence and conformity using other­regarding preference models derived from heterodox economics. To address this research gap, the present study investigates the neuropsychological underpinnings of social contagion by utilizing event­related potentials (ERPs) recorded while subjects engage in mini­dictator games. The behavioral analysis revealed that contagion had an impact on the participants' preferences, leading to a change in their choices. We observed a P300 component in the midline and right posterior during the time window of 200­350 ms after stimulus onset, which showed a significant increase in mean amplitude when participants observed others' behavior, compared to when they made decisions based on their own preferences. Moreover, the lack of late positive potential in the time window of 500­650 ms suggests that the presence of P300 may indicate difficulty in making decisions. In summary, by analyzing both behavioral and ERP data, this study may provide a more comprehensive understanding of the cognitive and neural processes that drive conformity and contagion behavior. Our analysis has the potential to inform policymakers in developing effective interventions for promoting positive social behaviors and reducing negative ones.

My choice, my actions: self-determination, not instrumental value of outcomes enhances outcome monitoring during learning.

Freedom of choice enhances our sense of agency. During goal-directed behavior, the freedom to choose between different response options increases the neural processing of positive and negative feedback, indicating enhanced outcome monitoring under conditions of high agency experience. However, it is unclear whether this enhancement is predominantly driven by an increased salience of self- compared to externally determined action outcomes or whether differences in the perceived instrumental value of outcomes contribute to outcome monitoring in goal-directed tasks. To test this, we recorded electroencephalography while participants performed a reinforcement learning task involving free choices, action-relevant forced choices, and action-irrelevant forced choices. We observed larger midfrontal theta power and N100 amplitudes for feedback following free choices compared with action-relevant and action-irrelevant forced choices. In addition, a Reward Positivity was only present for free but not forced choice outcomes. Crucially, our results indicate that enhanced outcome processing is not driven by the relevance of outcomes for future actions but rather stems from the association of outcomes with recent self-determined choice. Our findings highlight the pivotal role of self-determination in tracking the consequences of our actions and contribute to an understanding of the cognitive processes underlying the choice-induced facilitation in outcome monitoring.

The right posterior parietal cortex mediates spatial reorienting of attentional choice bias.

Attention facilitates behavior by enhancing perceptual sensitivity (sensory processing) and choice bias (decisional weighting) for attended information. Whether distinct neural substrates mediate these distinct components of attention remains unknown. We investigate the causal role of key nodes of the right posterior parietal cortex (rPPC) in the forebrain attention network in sensitivity versus bias control. Two groups of participants performed a cued attention task while we applied either inhibitory, repetitive transcranial magnetic stimulation (n = 28) or 40 Hz transcranial alternating current stimulation (n = 26) to the dorsal rPPC. We show that rPPC stimulation - with either modality - impairs task performance by selectively altering attentional modulation of bias but not sensitivity. Specifically, participants' bias toward the uncued, but not the cued, location reduced significantly following rPPC stimulation - an effect that was consistent across both neurostimulation cohorts. In sum, the dorsal rPPC causally mediates the reorienting of choice bias, one particular component of visual spatial attention.

Attentional dynamics of evidence accumulation explain why more numerate people make better decisions under risk.

In decisions under risk, more numerate people are typically more likely to choose the option with the highest expected value (EV) than less numerate ones. Prior research indicates that this finding cannot be explained by differences in the reliance on explicit EV calculation. The current work uses the attentional Drift Diffusion Model as a unified computational framework to formalize three candidate mechanisms of pre-decisional information search and processing-namely, attention allocation, amount of deliberation, and distorted processing of value-which may differ between more and less numerate people and explain differences in decision quality. Computational modeling of an eye-tracking experiment on risky choice demonstrates that numeracy is linked to how people allocate their attention across the options, how much evidence they require before committing to a choice, and also how strongly they distort currently non-attended information during preference formation. Together, especially the latter two mechanisms largely mediate the effect of numeracy on decision quality. Overall, the current work disentangles and quantifies latent aspects of the dynamics of preference formation, explicates how their interplay may give rise to manifest differences in decision quality, and thereby provides a fully formalized, mechanistic explanation for the link between numeracy and decision quality in risky choice.

Increased color preference through the introduction of luminance noise in chromatic stimuli.

Humans exhibit consistent color preferences that are often described as a curvilinear pattern across hues. The recent literature posits that color preference is linked to the preference for objects or other entities associated with those colors. However, many studies examine this preference using isoluminant colors, which don't reflect the natural viewing experience typically influenced by different light intensities. The inclusion of random luminance levels (luminance noise) in chromatic stimuli may provide an initial step towards assessing color preference as it is presented in the real world. Employing mosaic stimuli, this study aimed to evaluate the influence of luminance noise on human color preference. Thirty normal trichromats engaged in a two-alternative forced-choice paradigm, indicating their color preferences between presented pairs. The chromatic stimuli included saturated versions of 8 standard hues, presented in mosaics with varying diameters under different luminance noise conditions. Results indicated that the inclusion of luminance noise increased color preference across all hues, specifically under the high luminance noise range, while the curvilinear pattern remained unchanged. Finally, women exhibit a greater sensitivity to the presence of luminance noise than men, potentially due to differences between men and women in aesthetic evaluation strategies.