Hierarchical inference as a source of human biases.

The finding that human decision-making is systematically biased continues to have an immense impact on both research and policymaking. Prevailing views ascribe biases to limited computational resources, which require humans to resort to less costly resource-rational heuristics. Here, we propose that many biases in fact arise due to a computationally costly way of coping with uncertainty-namely, hierarchical inference-which by nature incorporates information that can seem irrelevant. We show how, in uncertain situations, Bayesian inference may avail of the environment's hierarchical structure to reduce uncertainty at the cost of introducing bias. We illustrate how this account can explain a range of familiar biases, focusing in detail on the halo effect and on the neglect of base rates. In each case, we show how a hierarchical-inference account takes the characterization of a bias beyond phenomenological description by revealing the computations and assumptions it might reflect. Furthermore, we highlight new predictions entailed by our account concerning factors that could mitigate or exacerbate bias, some of which have already garnered empirical support. We conclude that a hierarchical inference account may inform scientists and policy makers with a richer understanding of the adaptive and maladaptive aspects of human decision-making.

Disrupted state transition learning as a computational marker of compulsivity.

BACKGROUND: Disorders involving compulsivity, fear, and anxiety are linked to beliefs that the world is less predictable. We lack a mechanistic explanation for how such beliefs arise. Here, we test a hypothesis that in people with compulsivity, fear, and anxiety, learning a probabilistic mapping between actions and environmental states is compromised. METHODS: In Study 1 (n = 174), we designed a novel online task that isolated state transition learning from other facets of learning and planning. To determine whether this impairment is due to learning that is too fast or too slow, we estimated state transition learning rates by fitting computational models to two independent datasets, which tested learning in environments in which state transitions were either stable (Study 2: n = 1413) or changing (Study 3: n = 192). RESULTS: Study 1 established that individuals with higher levels of compulsivity are more likely to demonstrate an impairment in state transition learning. Preliminary evidence here linked this impairment to a common factor comprising compulsivity and fear. Studies 2 and 3 showed that compulsivity is associated with learning that is too fast when it should be slow (i.e. when state transition are stable) and too slow when it should be fast (i.e. when state transitions change). CONCLUSIONS: Together, these findings indicate that compulsivity is associated with a dysregulation of state transition learning, wherein the rate of learning is not well adapted to the task environment. Thus, dysregulated state transition learning might provide a key target for therapeutic intervention in compulsivity.

If you don't let it in, you don't have to get it out: Thought preemption as a method to control unwanted thoughts.

To attain goals, people must proactively prevent interferences and react to interferences once they occur. Whereas most research focuses on how people deal with external interferences, here we investigate the use of proactive and reactive control in dealing with unwanted thoughts. To examine this question, we asked people to generate an association to each of several repeating cue words, while forbidding the repetition of associations. Reactively rejecting and replacing unwanted repeated associations after they occur entails slower response times. Conversely, proactive control entails constricting the search space and thus faster response times. To gain further insight into different potential proactive thought control mechanisms, we augmented the analysis of raw response times with a novel, hypothesis-based, tractable computational model describing how people serially sample associations. Our results indicate that people primarily react to unwanted thoughts after they occur. Yet, we found evidence for two latent proactive control mechanisms: one that allows people to mitigate the episodic strengthening of repeated thoughts, and another that helps avoid looping in a repetitive thought. Exploratory analysis showed a relationship between model parameters and self-reported individual differences in the control over unwanted thoughts in daily life. The findings indicate the novel task and model can advance our understanding of how people can and cannot control their thoughts and memories, and benefit future research on the mechanisms responsible for unwanted thought in different psychiatric conditions. Finally, we discuss implications concerning the involvement of associative thinking and various control processes in semantic fluency, decision-making and creativity.

Optimism and pessimism in optimised replay.

The replay of task-relevant trajectories is known to contribute to memory consolidation and improved task performance. A wide variety of experimental data show that the content of replayed sequences is highly specific and can be modulated by reward as well as other prominent task variables. However, the rules governing the choice of sequences to be replayed still remain poorly understood. One recent theoretical suggestion is that the prioritization of replay experiences in decision-making problems is based on their effect on the choice of action. We show that this implies that subjects should replay sub-optimal actions that they dysfunctionally choose rather than optimal ones, when, by being forgetful, they experience large amounts of uncertainty in their internal models of the world. We use this to account for recent experimental data demonstrating exactly pessimal replay, fitting model parameters to the individual subjects' choices.

Training diversity promotes absolute-value-guided choice.

Many decision-making studies have demonstrated that humans learn either expected values or relative preferences among choice options, yet little is known about what environmental conditions promote one strategy over the other. Here, we test the novel hypothesis that humans adapt the degree to which they form absolute values to the diversity of the learning environment. Since absolute values generalize better to new sets of options, we predicted that the more options a person learns about the more likely they would be to form absolute values. To test this, we designed a multi-day learning experiment comprising twenty learning sessions in which subjects chose among pairs of images each associated with a different probability of reward. We assessed the degree to which subjects formed absolute values and relative preferences by asking them to choose between images they learned about in separate sessions. We found that concurrently learning about more images within a session enhanced absolute-value, and suppressed relative-preference, learning. Conversely, cumulatively pitting each image against a larger number of other images across multiple sessions did not impact the form of learning. These results show that the way humans encode preferences is adapted to the diversity of experiences offered by the immediate learning context.

The empathic personality profile: Using personality characteristics to reveal genetic, environmental, and developmental patterns of adolescents' empathy.

OBJECTIVE: How do genetic and environmental processes affect empathy during early adolescence? This study illuminated this question by examining the aetiology of empathy with the aetiology of other personality characteristics. METHOD: Israeli twin adolescents rated their empathy and personality at ages 11 (N = 1176) and 13 (N = 821) (733 families, 51.4% females). Parents rated adolescents' emotional empathy. Adolescents performed an emotion recognition task, indicating cognitive empathy. RESULTS: Using a cross-validated statistical learning algorithm, this study found emotional and cognitive "empathic personality profiles," which describe and predict self-reported empathy from nuanced Big-Five personality characteristics, or "nuances" (i.e., individual items). These profiles predicted empathy moderately (R2  = 0.17-0.24) and were stable and robust, within each age and between ages. They also predicted empathy in a new sample of older nontwin adolescents (N = 96) and were validated against non-self-report empathy measures. Both emotional and cognitive empathy were predicted by nuances representing positive attitudes toward others, trust, forgiveness, and openness to experiences. Emotional empathy was also predicted by nuances representing anxiousness and negative reactivity. Twin analyses revealed overlapping genetic and environmental influences on empathy and the empathic personality profiles and overlapping environmental influences on empathy-personality change. CONCLUSIONS: This study demonstrates how addressing the complexity of individuals' personalities can inform adolescents' empathy development.

Coping with emotional dysregulation among young adults with ADHD: A mixed-method study of self-awareness and strategies in daily life.

BACKGROUND: Emotional dysregulation (ED) impacts functional outcomes among individuals with attention-deficit hyperactivity disorder (ADHD). Self-awareness and strategies may enhance coping with ED yet are rarely studied in ADHD. OBJECTIVES: To explore ED-related self-awareness and strategies in daily life of adults with ADHD, and to examine the interrelations between them and their association with symptoms. METHODS: Sixty young adults with ADHD participated in a mixed-method study. At baseline, self-awareness and strategies were assessed using the Self-Regulation Skills Interview (SRSI); ADHD symptoms were self-rated using the ASRS symptom checklist. Then, symptoms were rated over 5-days using ecological momentary assessment (EMA). RESULTS: Significant challenges in self-awareness and strategies were demonstrated quantitatively and qualitatively. Awareness of ED was associated with variability of ADHD symptoms on EMA yet not with symptom severity. Qualitative content analysis revealed a range of self-awareness levels, which were related to noticing ED-related cues and understanding contextual factors predictive of ED. Self-awareness and strategies were significantly associated. Strategies varied regarding effort, individual preference and temporality. CONCLUSIONS: Variability of ADHD symptoms was negatively associated with self-awareness of ED. Strategy selection in daily-life among adults with ADHD may be affected by self-awareness and by a possible trade-off between short-term effort and long-term effectiveness.

Doubting what you already know: Uncertainty regarding state transitions is associated with obsessive compulsive symptoms.

Obsessive compulsive (OC) symptoms involve excessive information gathering (e.g., checking, reassurance-seeking), and uncertainty about possible, often catastrophic, future events. Here we propose that these phenomena are the result of excessive uncertainty regarding state transitions (transition uncertainty): a computational impairment in Bayesian inference leading to a reduced ability to use the past to predict the present and future, and to oversensitivity to feedback (i.e. prediction errors). Using a computational model of Bayesian learning under uncertainty in a reversal learning task, we investigate the relationship between OC symptoms and transition uncertainty. Individuals high and low in OC symptoms performed a task in which they had to detect shifts (i.e. transitions) in cue-outcome contingencies. Modeling subjects' choices was used to estimate each individual participant's transition uncertainty and associated responses to feedback. We examined both an optimal observer model and an approximate Bayesian model in which participants were assumed to attend (and learn about) only one of several cues on each trial. Results suggested the participants were more likely to distribute attention across cues, in accordance with the optimal observer model. As hypothesized, participants with higher OC symptoms exhibited increased transition uncertainty, as well as a pattern of behavior potentially indicative of a difficulty in relying on learned contingencies, with no evidence for perseverative behavior. Increased transition uncertainty compromised these individuals' ability to predict ensuing feedback, rendering them more surprised by expected outcomes. However, no evidence for excessive belief updating was found. These results highlight a potential computational basis for OC symptoms and obsessive compulsive disorder (OCD). The fact the OC symptoms predicted a decreased reliance on the past rather than perseveration challenges preconceptions of OCD as a disorder of inflexibility. Our results have implications for the understanding of the neurocognitive processes leading to excessive uncertainty and distrust of past experiences in OCD.

Emotional dysregulation and health related quality of life in young adults with ADHD: a cross sectional study.

OBJECTIVE: Attention Deficit Hyperactivity Disorder (ADHD) is associated with emotional dysregulation (ED) and impaired health related quality of life (HRQoL). However, the role of ED in explaining the relationship between ADHD and HRQoL is unclear. The purpose of the present study was to do so in a sample of non-referred young adults with and without ADHD. METHOD: The study design was cross-sectional. A non-clinical sample of 63 young adults with ADHD (mean age = 24.86 years, SD = 3.25, 78% university students) and 69 gender-matched controls (mean age = 23.84 years, SD = 2.59, 89% university students) were recruited. The Adult ADHD Quality-of-Life scale was used to measure HRQoL; The Self-Report Wender-Reimherr Adult Attention Deficit Disorder Scale and the Difficulties in Emotion Regulation Scale were used to measure ED. Group differences on all measures were tested using univariate and multivariate analyses of covariance, while controlling for age. Finally, a moderation analysis was used in order to examine the impact of ED on HRQoL beyond that accounted for by ADHD symptoms. RESULTS: Both HRQoL and ED were significantly worse for the ADHD group compared to the control group. The medication status of the ADHD group participants had no significant effect on the level of ADHD symptoms, ED or HRQoL. ED moderated the effect of ADHD symptoms on HRQoL for the ADHD group. CONCLUSION: The findings support the centrality of ED in ADHD and its crucial influence on HRQoL. Young adults with ADHD and high levels of ED are at risk for aversive impact on their well-being regardless of their ADHD symptoms level.

Young adults with Attention Deficit Hyperactivity Disorder (ADHD) struggle with poor quality of life (QoL). Emotional regulation is one's ability to modify their emotional state to promote adaptive, goal-oriented behaviors. Emotional dysregulation is a common yet neglected feature of people with ADHD. Our results show that young adults with ADHD are twice more likely to suffer from emotional dysregulation then their peers. Moreover, higher levels of emotional dysregulation predicted lower levels of QoL. These findings support the centrality of ED in ADHD and their crucial influence on everyday QoL. These findings are important not only on the theoretical level, but may also contribute to the design of interventions that aim to promote quality of life.

Distinct Roles of Dopamine and Noradrenaline in Incidental Memory.

Episodic memory is sensitive to the influence of neuromodulators, such as dopamine and noradrenaline. These influences are considered important in the expression of several known memory biases, though their specific role in memory remains unclear. Using pharmacological agents with relatively high selectivity for either dopamine (400 mg amisulpride) or noradrenaline (40 mg propranolol) we examined their specific contribution to incidental memory. In a double-blind placebo-controlled human study (30 females, 30 males in total), we show that a memory selectivity bias was insensitive to propranolol but sensitive to amisulpride, consistent with a dominant influence from dopamine. By contrast, a putative arousal-induced memory boosting effect was insensitive to amisulpride but was sensitive to propranolol, consistent with a dominant noradrenaline effect. Thus, our findings highlight specific functional roles for dopamine and noradrenaline neurotransmission in the expression of incidental memory.SIGNIFICANCE STATEMENT Why some information is preferentially encoded into memory while other information is not is a central question in cognitive neuroscience. The neurotransmitters dopamine and noradrenaline are often assumed critical in influencing this selectivity, but their specific contributions remain obscure. In this double-blind, placebo-controlled, between-subjects drug study, we investigate the contributions of noradrenaline and dopamine to episodic memory. Using an incidental memory task, we find that blocking dopamine (400 mg amisulpride) eliminates a neural-gain related memory selectivity bias. Blocking noradrenaline function (40 mg propranolol), in contrast, abolishes an arousal-related memory enhancement. In this assessment of dopamine and noradrenaline neuromodulatory effects we reveal their specific contributions to episodic memory.

Towards formal models of psychopathological traits that explain symptom trajectories.

BACKGROUND: A dominant methodology in contemporary clinical neuroscience is the use of dimensional self-report questionnaires to measure features such as psychological traits (e.g., trait anxiety) and states (e.g., depressed mood). These dimensions are then mapped to biological measures and computational parameters. Researchers pursuing this approach tend to equate a symptom inventory score (plus noise) with some latent psychological trait. MAIN TEXT: We argue this approach implies weak, tacit, models of traits that provide fixed predictions of individual symptoms, and thus cannot account for symptom trajectories within individuals. This problem persists because (1) researchers are not familiarized with formal models that relate internal traits to within-subject symptom variation and (2) rely on an assumption that trait self-report inventories accurately indicate latent traits. To address these concerns, we offer a computational model of trait depression that demonstrates how parameters instantiating a given trait remain stable while manifest symptom expression varies predictably. We simulate patterns of mood variation from both the computational model and the standard self-report model and describe how to quantify the relative validity of each model using a Bayesian procedure. CONCLUSIONS: Ultimately, we would urge a tempering of a reliance on self-report inventories and recommend a shift towards developing mechanistic trait models that can explain within-subject symptom dynamics.

Separate mesocortical and mesolimbic pathways encode effort and reward learning signals.

Optimal decision making mandates organisms learn the relevant features of choice options. Likewise, knowing how much effort we should expend can assume paramount importance. A mesolimbic network supports reward learning, but it is unclear whether other choice features, such as effort learning, rely on this same network. Using computational fMRI, we show parallel encoding of effort and reward prediction errors (PEs) within distinct brain regions, with effort PEs expressed in dorsomedial prefrontal cortex and reward PEs in ventral striatum. We show a common mesencephalic origin for these signals evident in overlapping, but spatially dissociable, dopaminergic midbrain regions expressing both types of PE. During action anticipation, reward and effort expectations were integrated in ventral striatum, consistent with a computation of an overall net benefit of a stimulus. Thus, we show that motivationally relevant stimulus features are learned in parallel dopaminergic pathways, with formation of an integrated utility signal at choice.

Striatal structure and function predict individual biases in learning to avoid pain.

Pain is an elemental inducer of avoidance. Here, we demonstrate that people differ in how they learn to avoid pain, with some individuals refraining from actions that resulted in painful outcomes, whereas others favor actions that helped prevent pain. These individual biases were best explained by differences in learning from outcome prediction errors and were associated with distinct forms of striatal responses to painful outcomes. Specifically, striatal responses to pain were modulated in a manner consistent with an aversive prediction error in individuals who learned predominantly from pain, whereas in individuals who learned predominantly from success in preventing pain, modulation was consistent with an appetitive prediction error. In contrast, striatal responses to success in preventing pain were consistent with an appetitive prediction error in both groups. Furthermore, variation in striatal structure, encompassing the region where pain prediction errors were expressed, predicted participants' predominant mode of learning, suggesting the observed learning biases may reflect stable individual traits. These results reveal functional and structural neural components underlying individual differences in avoidance learning, which may be important contributors to psychiatric disorders involving pathological harm avoidance behavior.

Catecholamine-Mediated Increases in Gain Enhance the Precision of Cortical Representations.

UNLABELLED: Neurophysiological evidence suggests that neuromodulators, such as norepinephrine and dopamine, increase neural gain in target brain areas. Computational models and prominent theoretical frameworks indicate that this should enhance the precision of neural representations, but direct empirical evidence for this hypothesis is lacking. In two functional MRI studies, we examine the effect of baseline catecholamine levels (as indexed by pupil diameter and manipulated pharmacologically) on the precision of object representations in the human ventral temporal cortex using angular dispersion, a powerful, multivariate metric of representational similarity (precision). We first report the results of computational model simulations indicating that increasing catecholaminergic gain should reduce the angular dispersion, and thus increase the precision, of object representations from the same category, as well as reduce the angular dispersion of object representations from distinct categories when distinct-category representations overlap. In Study 1 (N = 24), we show that angular dispersion covaries with pupil diameter, an index of baseline catecholamine levels. In Study 2 (N = 24), we manipulate catecholamine levels and neural gain using the norepinephrine transporter blocker atomoxetine and demonstrate consistent, causal effects on angular dispersion and brain-wide functional connectivity. Despite the use of very different methods of examining the effect of baseline catecholamine levels, our results show a striking convergence and demonstrate that catecholamines increase the precision of neural representations. SIGNIFICANCE STATEMENT: Norepinephrine and dopamine are among the most widely distributed and ubiquitous neuromodulators in the mammalian brain and have a profound and pervasive impact on cognition. Baseline catecholamine levels tend to increase with increasing task engagement in tasks involving perceptual decisions, yet there is currently no direct evidence of the specific impact of these increases in catecholamine levels on perceptual encoding. Our results fill this void by showing that catecholamines enhance the precision of encoding cortical object representations, and by suggesting that this effect is mediated by increases in neural gain, thus offering a mechanistic account of our key finding.

Do You See the Forest or the Tree? Neural Gain and Breadth Versus Focus in Perceptual Processing.

When perceiving rich sensory information, some people may integrate its various aspects, whereas other people may selectively focus on its most salient aspects. We propose that neural gain modulates the trade-off between breadth and selectivity, such that high gain focuses perception on those aspects of the information that have the strongest, most immediate influence, whereas low gain allows broader integration of different aspects. We illustrate our hypothesis using a neural-network model of ambiguous-letter perception. We then report an experiment demonstrating that, as predicted by the model, pupil-diameter indices of higher gain are associated with letter perception that is more selectively focused on the letter's shape or, if primed, its semantic content. Finally, we report a recognition-memory experiment showing that the relationship between gain and selective processing also applies when the influence of different stimulus features is voluntarily modulated by task demands.

Amplified selectivity in cognitive processing implements the neural gain model of norepinephrine function.

Previous work has suggested that an interaction between local selective (e.g., glutamatergic) excitation and global gain modulation (via norepinephrine) amplifies selectivity in information processing. Mather et al. extend this existing theory by suggesting that localized gain modulation may further mediate this effect - an interesting prospect that invites new theoretical and experimental work.

A cognitive-computational account of mood swings in adolescence.

Teenagers have a reputation for being fickle, in both their choices and their moods. This variability may help adolescents as they begin to independently navigate novel environments. Recently, however, adolescent moodiness has also been linked to psychopathology. Here, we consider adolescents' mood swings from a novel computational perspective, grounded in reinforcement learning (RL). This model proposes that mood is determined by surprises about outcomes in the environment, and how much we learn from these surprises. It additionally suggests that mood biases learning and choice in a bidirectional manner. Integrating independent lines of research, we sketch a cognitive-computational account of how adolescents' mood, learning, and choice dynamics influence each other, with implications for normative and psychopathological development.

The computational psychopathology of emotion.

Mood and anxiety disorders involve recurring, maladaptive patterns of distinct emotions and moods. Here, we argue that understanding these maladaptive patterns first requires understanding how emotions and moods guide adaptive behavior. We thus review recent progress in computational accounts of emotion that aims to explain the adaptive role of distinct emotions and mood. We then highlight how this emerging approach could be used to explain maladaptive emotions in various psychopathologies. In particular, we identify three computational factors that may be responsible for excessive emotions and moods of different types: self-intensifying affective biases, misestimations of predictability, and misestimations of controllability. Finally, we outline how the psychopathological roles of these factors can be tested, and how they may be used to improve psychotherapeutic and psychopharmacological interventions.

Emotions as computations.

Emotions ubiquitously impact action, learning, and perception, yet their essence and role remain widely debated. Computational accounts of emotion aspire to answer these questions with greater conceptual precision informed by normative principles and neurobiological data. We examine recent progress in this regard and find that emotions may implement three classes of computations, which serve to evaluate states, actions, and uncertain prospects. For each of these, we use the formalism of reinforcement learning to offer a new formulation that better accounts for existing evidence. We then consider how these distinct computations may map onto distinct emotions and moods. Integrating extensive research on the causes and consequences of different emotions suggests a parsimonious one-to-one mapping, according to which emotions are integral to how we evaluate outcomes (pleasure & pain), learn to predict them (happiness & sadness), use them to inform our (frustration & content) and others' (anger & gratitude) actions, and plan in order to realize (desire & hope) or avoid (fear & anxiety) uncertain outcomes.

Computational Mechanisms of Addiction and Anxiety: A Developmental Perspective.

A central goal of computational psychiatry is to identify systematic relationships between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. Most psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.