From likely to likable: The role of statistical typicality in human social assessment of faces.

Humans readily form social impressions, such as attractiveness and trustworthiness, from a stranger's facial features. Understanding the provenance of these impressions has clear scientific importance and societal implications. Motivated by the efficient coding hypothesis of brain representation, as well as Claude Shannon's theoretical result that maximally efficient representational systems assign shorter codes to statistically more typical data (quantified as log likelihood), we suggest that social "liking" of faces increases with statistical typicality. Combining human behavioral data and computational modeling, we show that perceived attractiveness, trustworthiness, dominance, and valence of a face image linearly increase with its statistical typicality (log likelihood). We also show that statistical typicality can at least partially explain the role of symmetry in attractiveness perception. Additionally, by assuming that the brain focuses on a task-relevant subset of facial features and assessing log likelihood of a face using those features, our model can explain the "ugliness-in-averageness" effect found in social psychology, whereby otherwise attractive, intercategory faces diminish in attractiveness during a categorization task.

Bayesian neural adjustment of inhibitory control predicts emergence of problem stimulant use.

Bayesian ideal observer models quantify individuals' context- and experience-dependent beliefs and expectations about their environment, which provides a powerful approach (i) to link basic behavioural mechanisms to neural processing; and (ii) to generate clinical predictors for patient populations. Here, we focus on (ii) and determine whether individual differences in the neural representation of the need to stop in an inhibitory task can predict the development of problem use (i.e. abuse or dependence) in individuals experimenting with stimulants. One hundred and fifty-seven non-dependent occasional stimulant users, aged 18-24, completed a stop-signal task while undergoing functional magnetic resonance imaging. These individuals were prospectively followed for 3 years and evaluated for stimulant use and abuse/dependence symptoms. At follow-up, 38 occasional stimulant users met criteria for a stimulant use disorder (problem stimulant users), while 50 had discontinued use (desisted stimulant users). We found that those individuals who showed greater neural responses associated with Bayesian prediction errors, i.e. the difference between actual and expected need to stop on a given trial, in right medial prefrontal cortex/anterior cingulate cortex, caudate, anterior insula, and thalamus were more likely to exhibit problem use 3 years later. Importantly, these computationally based neural predictors outperformed clinical measures and non-model based neural variables in predicting clinical status. In conclusion, young adults who show exaggerated brain processing underlying whether to 'stop' or to 'go' are more likely to develop stimulant abuse. Thus, Bayesian cognitive models provide both a computational explanation and potential predictive biomarkers of belief processing deficits in individuals at risk for stimulant addiction.

Altered neural processing of the need to stop in young adults at risk for stimulant dependence.

Identification of neurocognitive predictors of substance dependence is an important step in developing approaches to prevent addiction. Given evidence of inhibitory control deficits in substance abusers (Monterosso et al., 2005; Fu et al., 2008; Lawrence et al., 2009; Tabibnia et al., 2011), we examined neural processing characteristics in human occasional stimulant users (OSU), a population at risk for dependence. A total of 158 nondependent OSU and 47 stimulant-naive control subjects (CS) were recruited and completed a stop signal task while undergoing functional magnetic resonance imaging (fMRI). A Bayesian ideal observer model was used to predict probabilistic expectations of inhibitory demand, P(stop), on a trial-to-trial basis, based on experienced trial history. Compared with CS, OSU showed attenuated neural activation related to P(stop) magnitude in several areas, including left prefrontal cortex and left caudate. OSU also showed reduced neural activation in the dorsal anterior cingulate cortex (dACC) and right insula in response to an unsigned Bayesian prediction error representing the discrepancy between stimulus outcome and the predicted probability of a stop trial. These results indicate that, despite minimal overt behavioral manifestations, OSU use fewer brain processing resources to predict and update the need for response inhibition, processes that are critical for adjusting and optimizing behavioral performance, which may provide a biomarker for the development of substance dependence.

Bayesian prediction and evaluation in the anterior cingulate cortex.

The dorsal anterior cingulate cortex (dACC) has been implicated in a variety of cognitive control functions, among them the monitoring of conflict, error, and volatility, error anticipation, reward learning, and reward prediction errors. In this work, we used a Bayesian ideal observer model, which predicts trial-by-trial probabilistic expectation of stop trials and response errors in the stop-signal task, to differentiate these proposed functions quantitatively. We found that dACC hemodynamic response, as measured by functional magnetic resonance imaging, encodes both the absolute prediction error between stimulus expectation and outcome, and the signed prediction error related to response outcome. After accounting for these factors, dACC has no residual correlation with conflict or error likelihood in the stop-signal task. Consistent with recent monkey neural recording studies, and in contrast with other neuroimaging studies, our work demonstrates that dACC reports at least two different types of prediction errors, and beyond contexts that are limited to reward processing.

Revisiting the Role of Uncertainty-Driven Exploration in a (Perceived) Non-Stationary World.

Humans are often faced with an exploration-versus-exploitation trade-off. A commonly used paradigm, multi-armed bandit, has shown humans to exhibit an "uncertainty bonus", which combines with estimated reward to drive exploration. However, previous studies often modeled belief updating using either a Bayesian model that assumed the reward contingency to remain stationary, or a reinforcement learning model. Separately, we previously showed that human learning in the bandit task is best captured by a dynamic-belief Bayesian model. We hypothesize that the estimated uncertainty bonus may depend on which learning model is employed. Here, we re-analyze a bandit dataset using all three learning models. We find that the dynamic-belief model captures human choice behavior best, while also uncovering a much larger uncertainty bonus than the other models. More broadly, our results also emphasize the importance of an appropriate learning model, as it is crucial for correctly characterizing the processes underlying human decision making.

Distinct motivations to seek out information in healthy individuals and problem gamblers.

As massive amounts of information are becoming available to people, understanding the mechanisms underlying information-seeking is more pertinent today than ever. In this study, we investigate the underlying motivations to seek out information in healthy and addicted individuals. We developed a novel decision-making task and a novel computational model which allows dissociating the relative contribution of two motivating factors to seek out information: a desire for novelty and a general desire for knowledge. To investigate whether/how the motivations to seek out information vary between healthy and addicted individuals, in addition to healthy controls we included a sample of individuals with gambling disorder-a form of addiction without the confound of substance consumption and characterized by compulsive gambling. Our results indicate that healthy subjects and problem gamblers adopt distinct information-seeking "modes". Healthy information-seeking behavior was mostly motivated by a desire for novelty. Problem gamblers, on the contrary, displayed reduced novelty-seeking and an increased desire for accumulating knowledge compared to healthy controls. Our findings not only shed new light on the motivations driving healthy and addicted individuals to seek out information, but they also have important implications for the treatment and diagnosis of behavioral addiction.

Adaptive behavior: humans act as Bayesian learners.

Subjects adapt to environmental changes differently depending on the perceived frequency of the changes--the environmental volatility--similar to an ideal Bayesian learner. This volatility information correlates with the fMRI BOLD signal in the anterior cingulate cortex.

Leveraging Computer Vision Face Representation to Understand Human Face Representation.

Face processing plays a critical role in human social life, from differentiating friends from enemies to choosing a life mate. In this work, we leverage various computer vision techniques, combined with human assessments of similarity between pairs of faces, to investigate human face representation. We find that combining a shape- and texture-feature based model (Active Appearance Model) with a particular form of metric learning, not only achieves the best performance in predicting human similarity judgments on held-out data (both compared to other algorithms and to humans), but also performs better or comparable to alternative approaches in modeling human social trait judgment (e.g. trustworthiness, attractiveness) and affective assessment (e.g. happy, angry, sad). This analysis yields several scientific findings: (1) facial similarity judgments rely on a relative small number of facial features (8-12), (2) race- and gender-informative features play a prominent role in similarity perception, (3) similarity-relevant features alone are insufficient to capture human face representation, in particular some affective features missing from similarity judgments are also necessary for constructing the complete psychological face representation.

Devaluation of Unchosen Options: A Bayesian Account of the Provenance and Maintenance of Overly Optimistic Expectations.

Humans frequently overestimate the likelihood of desirable events while underestimating the likelihood of undesirable ones: a phenomenon known as unrealistic optimism. Previously, it was suggested that unrealistic optimism arises from asymmetric belief updating, with a relatively reduced coding of undesirable information. Prior studies have shown that a reinforcement learning (RL) model with asymmetric learning rates (greater for a positive prediction error than a negative prediction error) could account for unrealistic optimism in a bandit task, in particular the tendency of human subjects to persistently choosing a single option when there are multiple equally good options. Here, we propose an alternative explanation of such persistent behavior, by modeling human behavior using a Bayesian hidden Markov model, the Dynamic Belief Model (DBM). We find that DBM captures human choice behavior better than the previously proposed asymmetric RL model. Whereas asymmetric RL attains a measure of optimism by giving better-than-expected outcomes higher learning weights compared to worse-than-expected outcomes, DBM does so by progressively devaluing the unchosen options, thus placing a greater emphasis on choice history independent of reward outcome (e.g. an oft-chosen option might continue to be preferred even if it has not been particularly rewarding), which has broadly been shown to underlie sequential effects in a variety of behavioral settings. Moreover, previous work showed that the devaluation of unchosen options in DBM helps to compensate for a default assumption of environmental non-stationarity, thus allowing the decision-maker to both be more adaptive in changing environments and still obtain near-optimal performance in stationary environments. Thus, the current work suggests both a novel rationale and mechanism for persistent behavior in bandit tasks.

Uncertainty, neuromodulation, and attention.

Uncertainty in various forms plagues our interactions with the environment. In a Bayesian statistical framework, optimal inference and prediction, based on unreliable observations in changing contexts, require the representation and manipulation of different forms of uncertainty. We propose that the neuromodulators acetylcholine and norepinephrine play a major role in the brain's implementation of these uncertainty computations. Acetylcholine signals expected uncertainty, coming from known unreliability of predictive cues within a context. Norepinephrine signals unexpected uncertainty, as when unsignaled context switches produce strongly unexpected observations. These uncertainty signals interact to enable optimal inference and learning in noisy and changeable environments. This formulation is consistent with a wealth of physiological, pharmacological, and behavioral data implicating acetylcholine and norepinephrine in specific aspects of a range of cognitive processes. Moreover, the model suggests a class of attentional cueing tasks that involve both neuromodulators and shows how their interactions may be part-antagonistic, part-synergistic.

Dynamics of attentional selection under conflict: toward a rational Bayesian account.

The brain exhibits remarkable facility in exerting attentional control in most circumstances, but it also suffers apparent limitations in others. The authors' goal is to construct a rational account for why attentional control appears suboptimal under conditions of conflict and what this implies about the underlying computational principles. The formal framework used is based on Bayesian probability theory, which provides a convenient language for delineating the rationale and dynamics of attentional selection. The authors illustrate these issues with the Eriksen flanker task, a classical paradigm that explores the effects of competing sensory inputs on response tendencies. The authors show how 2 distinctly formulated models, based on compatibility bias and spatial uncertainty principles, can account for the behavioral data. They also suggest novel experiments that may differentiate these models. In addition, they elaborate a simplified model that approximates optimal computation and may map more directly onto the underlying neural machinery. This approximate model uses conflict monitoring, putatively mediated by the anterior cingulate cortex, as a proxy for compatibility representation. The authors also consider how this conflict information might be disseminated and used to control processing.

Bayesian computational markers of relapse in methamphetamine dependence.

Methamphetamine use disorder is associated with a high likelihood of relapse. Identifying robust predictors of relapse that have explanatory power is critical to develop secondary prevention based on a mechanistic understanding of relapse. Computational approaches have the potential to identify such predictive markers of psychiatric illness, with the advantage of providing a finer mechanistic explanation of the cognitive processes underlying psychiatric vulnerability. In this study, sixty-two recently sober methamphetamine-dependent individuals were recruited from a 28-day inpatient treatment program, and completed a Stop Signal Task (SST) while undergoing functional magnetic resonance imaging (fMRI). These individuals were prospectively followed for 1 year and assessed for relapse to methamphetamine use. Thirty-three percent of followed participants reported relapse. We found that neural activity associated with two types of Bayesian prediction error, i.e. the difference between actual and expected need to stop on a given trial, significantly differentiated those individuals who remained abstinent and those who relapsed. Specifically, relapsed individuals exhibited smaller neural activations to such Bayesian prediction errors relative to those individuals who remained abstinent in the left temporoparietal junction (Cohen's d = 0.91), the left inferior frontal gyrus (Cohen's d = 0.57), and left anterior insula (Cohen's d = 0.63). In contrast, abstinent and relapsed participants did not differ in neural activation to non-model based task contrasts or on various self-report clinical measures. In conclusion, Bayesian cognitive models may help identify predictive biomarkers of relapse, while providing a computational explanation of belief processing and updating deficits in individuals with methamphetamine use disorder.

Demystifying excessively volatile human learning: A Bayesian persistent prior and a neural approximation.

Understanding how humans and animals learn about statistical regularities in stable and volatile environments, and utilize these regularities to make predictions and decisions, is an important problem in neuroscience and psychology. Using a Bayesian modeling framework, specifically the Dynamic Belief Model (DBM), it has previously been shown that humans tend to make the default assumption that environmental statistics undergo abrupt, unsignaled changes, even when environmental statistics are actually stable. Because exact Bayesian inference in this setting, an example of switching state space models, is computationally intensive, a number of approximately Bayesian and heuristic algorithms have been proposed to account for learning/prediction in the brain. Here, we examine a neurally plausible algorithm, a special case of leaky integration dynamics we denote as EXP (for exponential filtering), that is significantly simpler than all previously suggested algorithms except for the delta-learning rule, and which far outperforms the delta rule in approximating Bayesian prediction performance. We derive the theoretical relationship between DBM and EXP, and show that EXP gains computational efficiency by foregoing the representation of inferential uncertainty (as does the delta rule), but that it nevertheless achieves near-Bayesian performance due to its ability to incorporate a "persistent prior" influence unique to DBM and absent from the other algorithms. Furthermore, we show that EXP is comparable to DBM but better than all other models in reproducing human behavior in a visual search task, suggesting that human learning and prediction also incorporates an element of persistent prior. More broadly, our work demonstrates that when observations are information-poor, detecting changes or modulating the learning rate is both difficult and thus unnecessary for making Bayes-optimal predictions.

Publisher Correction: Learning the value of information and reward over time when solving exploration-exploitation problems.

A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has been fixed in the paper.

Motor Preparation Disrupts Proactive Control in the Stop Signal Task.

In a study of the stop signal task (SST) we employed Bayesian modeling to compute the estimated likelihood of stop signal or P(Stop) trial by trial and identified regional processes of conflict anticipation and response slowing. A higher P(Stop) is associated with prolonged go trial reaction time (goRT)-a form of sequential effect-and reflects proactive control of motor response. However, some individuals do not demonstrate a sequential effect despite similar go and stop success (SS) rates. We posited that motor preparation may disrupt proactive control more in certain individuals than others. Specifically, the time interval between trial and go signal onset-the fore-period (FP)-varies across trials and a longer FP is associated with a higher level of motor preparation and shorter goRT. Greater motor preparatory activities may disrupt proactive control. To test this hypothesis, we compared brain activations and Granger causal connectivities of 81 adults who demonstrated a sequential effect (SEQ) and 35 who did not (nSEQ). SEQ and nSEQ did not differ in regional activations to conflict anticipation, motor preparation, goRT slowing or goRT speeding. In contrast, SEQ and nSEQ demonstrated different patterns of Granger causal connectivities. P(Stop) and FP activations shared reciprocal influence in SEQ but FP activities Granger caused P(Stop) activities unidirectionally in nSEQ, and FP activities Granger caused goRT speeding activities in nSEQ but not SEQ. These findings support the hypothesis that motor preparation disrupts proactive control in nSEQ and provide direct neural evidence for interactive go and stop processes.

Learning the value of information and reward over time when solving exploration-exploitation problems.

To flexibly adapt to the demands of their environment, animals are constantly exposed to the conflict resulting from having to choose between predictably rewarding familiar options (exploitation) and risky novel options, the value of which essentially consists of obtaining new information about the space of possible rewards (exploration). Despite extensive research, the mechanisms that subtend the manner in which animals solve this exploitation-exploration dilemma are still poorly understood. Here, we investigate human decision-making in a gambling task in which the informational value of each trial and the reward potential were separately manipulated. To better characterize the mechanisms that underlined the observed behavioural choices, we introduce a computational model that augments the standard reward-based reinforcement learning formulation by associating a value to information. We find that both reward and information gained during learning influence the balance between exploitation and exploration, and that this influence was dependent on the reward context. Our results shed light on the mechanisms that underpin decision-making under uncertainty, and suggest new approaches for investigating the exploration-exploitation dilemma throughout the animal kingdom.

Anhedonia and anxiety underlying depressive symptomatology have distinct effects on reward-based decision-making.

Depressive pathology, which includes both heightened negative affect (e.g., anxiety) and reduced positive affect (e.g., anhedonia), is known to be associated with sub-optimal decision-making, particularly in uncertain environments. Here, we use a computational approach to quantify and disambiguate how individual differences in these affective measures specifically relate to different aspects of learning and decision-making in reward-based choice behavior. Fifty-three individuals with a range of depressed mood completed a two-armed bandit task, in which they choose between two arms with fixed but unknown reward rates. The decision-making component, which chooses among options based on current expectations about reward rates, is modeled by two different decision policies: a learning-independent Win-stay/Lose-shift (WSLS) policy that ignores all previous experiences except the last trial, and Softmax, which prefers the arm with the higher expected reward. To model the learning component for the Softmax choice policy, we use a Bayesian inference model, which updates estimated reward rates based on the observed history of trial outcomes. Softmax with Bayesian learning better fits the behavior of 55% of the participants, while the others are better fit by a learning-independent WSLS strategy. Among Softmax "users", those with higher anhedonia are less likely to choose the option estimated to be most rewarding. Moreover, the Softmax parameter mediates the inverse relationship between anhedonia and overall monetary gains. On the other hand, among WSLS "users", higher state anxiety correlates with increasingly better ability of WSLS, relative to Softmax, to explain subjects' trial-by-trial choices. In summary, there is significant variability among individuals in their reward-based, exploratory decision-making, and this variability is at least partly mediated in a very specific manner by affective attributes, such as hedonic tone and state anxiety.

Inseparability of Go and Stop in Inhibitory Control: Go Stimulus Discriminability Affects Stopping Behavior.

Inhibitory control, the ability to stop or modify preplanned actions under changing task conditions, is an important component of cognitive functions. Two lines of models of inhibitory control have previously been proposed for human response in the classical stop-signal task, in which subjects must inhibit a default go response upon presentation of an infrequent stop signal: (1) the race model, which posits two independent go and stop processes that race to determine the behavioral outcome, go or stop; and (2) an optimal decision-making model, which posits that observers decides whether and when to go based on continually (Bayesian) updated information about both the go and stop stimuli. In this work, we probe the relationship between go and stop processing by explicitly manipulating the discrimination difficulty of the go stimulus. While the race model assumes the go and stop processes are independent, and therefore go stimulus discriminability should not affect the stop stimulus processing, we simulate the optimal model to show that it predicts harder go discrimination should result in longer go reaction time (RT), lower stop error rate, as well as faster stop-signal RT. We then present novel behavioral data that validate these model predictions. The results thus favor a fundamentally inseparable account of go and stop processing, in a manner consistent with the optimal model, and contradicting the independence assumption of the race model. More broadly, our findings contribute to the growing evidence that the computations underlying inhibitory control are systematically modulated by cognitive influences in a Bayes-optimal manner, thus opening new avenues for interpreting neural responses underlying inhibitory control.

Reduced Neural Recruitment for Bayesian Adjustment of Inhibitory Control in Methamphetamine Dependence.

Delineating the processes that contribute to the progression and maintenance of substance dependence is critical to understanding and preventing addiction. Several previous studies have shown inhibitory control deficits in individuals with stimulant use disorder. We used a Bayesian computational approach to examine potential neural deficiencies in the dynamic predictive processing underlying inhibitory function among recently abstinent methamphetamine-dependent individuals (MDIs), a population at high risk of relapse. Sixty-two MDIs were recruited from a 28-day inpatient treatment program at the San Diego Veterans Affairs Medical Center and compared with 34 healthy control subjects. They completed a stop-signal task during functional magnetic resonance imaging. A Bayesian ideal observer model was used to predict individuals' trial-to-trial probabilistic expectations of inhibitory response, P(stop), to identify group differences specific to Bayesian expectation and prediction error computation. Relative to control subjects, MDIs were more likely to make stop errors on difficult trials and had attenuated slowing following stop errors. MDIs further exhibited reduced sensitivity as measured by the neural tracking of a Bayesian measure of surprise (unsigned prediction error), which was evident across all trials in the left posterior caudate and orbitofrontal cortex (Brodmann area 11), and selectively on stop error trials in the right thalamus and inferior parietal lobule. MDIs are less sensitive to surprising task events, both across trials and upon making commission errors, which may help explain why these individuals may not engage in switching strategy when the environment changes, leading to adverse consequences.

Statistical learning and adaptive decision-making underlie human response time variability in inhibitory control.

Response time (RT) is an oft-reported behavioral measure in psychological and neurocognitive experiments, but the high level of observed trial-to-trial variability in this measure has often limited its usefulness. Here, we combine computational modeling and psychophysics to examine the hypothesis that fluctuations in this noisy measure reflect dynamic computations in human statistical learning and corresponding cognitive adjustments. We present data from the stop-signal task (SST), in which subjects respond to a go stimulus on each trial, unless instructed not to by a subsequent, infrequently presented stop signal. We model across-trial learning of stop signal frequency, P(stop), and stop-signal onset time, SSD (stop-signal delay), with a Bayesian hidden Markov model, and within-trial decision-making with an optimal stochastic control model. The combined model predicts that RT should increase with both expected P(stop) and SSD. The human behavioral data (n = 20) bear out this prediction, showing P(stop) and SSD both to be significant, independent predictors of RT, with P(stop) being a more prominent predictor in 75% of the subjects, and SSD being more prominent in the remaining 25%. The results demonstrate that humans indeed readily internalize environmental statistics and adjust their cognitive/behavioral strategy accordingly, and that subtle patterns in RT variability can serve as a valuable tool for validating models of statistical learning and decision-making. More broadly, the modeling tools presented in this work can be generalized to a large body of behavioral paradigms, in order to extract insights about cognitive and neural processing from apparently quite noisy behavioral measures. We also discuss how this behaviorally validated model can then be used to conduct model-based analysis of neural data, in order to help identify specific brain areas for representing and encoding key computational quantities in learning and decision-making.