The spread of affective and semantic valence representations across states.

In many decision problems, outcomes are not reached after a single action but rather after a series of events or states. To optimize decisions over multiple states, representations of how good or bad the outcomes are, that is, the outcomes' valence, should spread across states. One mechanism for valence spreading is a temporal, state-independent process in which a single valence representation is updated when an outcome is experienced and fades away afterwards. Each state's valence is based on its temporal proximity to the experienced outcome. An alternative, state-dependent mechanism relies on the structure of transitions between states, updating a separate valence representation for each state according to its spatial distance from the outcomes. We examined how these mechanistic accounts shape the spread of two formats of valence representation, feelings (affective valence) and knowledge (semantic valence), between states. In two pre-registered experiments (N = 585), we used a novel task in which participants move in a four-state maze, one of which contains an outcome. The participants provide self-reports of affective and semantic valence throughout the maze and after finishing it. Results show that the affective representation of negative valence is more localized in state-space than the semantic representation. We also found evidence for the relative reliance of the affective valence on a temporal, state-independent mechanism and of the semantic valence on a structured, state-dependent mechanism. Our findings provide mechanistic accounts for the differences between affective and semantic valence representations and indicate how such representations may play a role in associative learning and decision-making.

Disentangling the contribution of individual and social learning processes in human advice-taking behavior.

The study of social learning examines how individuals learn from others by means of observation, imitation, or compliance with advice. However, it still remains largely unknown whether social learning processes have a distinct contribution to behavior, independent from non-social trial-and-error learning that often occurs simultaneously. 153 participants completed a reinforcement learning task, where they were asked to make choices to gain rewards. Advice from an artificial teacher was presented in 60% of the trials, allowing us to compare choice behavior with and without advice. Results showed a strong and reliable tendency to follow advice (test-retest reliability ~0.73). Computational modeling suggested a unique contribution of three distinct learning strategies: (a) individual learning (i.e., learning the value of actions, independent of advice), (b) informed advice-taking (i.e., learning the value of following advice), and (c) non-informed advice-taking (i.e., a constant bias to follow advice regardless of outcome history). Comparing artificial and empirical data provided specific behavioral regression signatures to both informed and non-informed advice taking processes. We discuss the theoretical implications of integrating internal and external information during the learning process.

Outcome context-dependence is not WEIRD: Comparing reinforcement- and description-based economic preferences worldwide.

Recent evidence indicates that reward value encoding in humans is highly context-dependent, leading to suboptimal decisions in some cases. But whether this computational constraint on valuation is a shared feature of human cognition remains unknown. To address this question, we studied the behavior of individuals from across 11 countries of markedly different socioeconomic and cultural makeup using an experimental approach that reliably captures context effects in reinforcement learning. Our findings show that all samples presented evidence of similar sensitivity to context. Crucially, suboptimal decisions generated by context manipulation were not explained by risk aversion, as estimated through a separate description-based choice task (i.e., lotteries) consisting of matched decision offers. Conversely, risk aversion significantly differed across countries. Overall, our findings suggest that context-dependent reward value encoding is a hardcoded feature of human cognition, while description-based decision-making is significantly sensitive to cultural factors.

Temporal dynamics of the semantic versus affective representations of valence during reversal learning.

Valence, the representation of a stimulus in terms of good or bad, plays a central role in models of affect, value-based learning theories, and value-based decision-making models. Previous work used Unconditioned Stimulus (US) to support a theoretical division between two different types of valence representations for a stimulus: the semantic representation of valence, i.e., stored accumulated knowledge about the value of the stimulus, and the affective representation of valence, i.e., the valence of the affective response to this stimulus. The current work extended past research by using a neutral Conditioned Stimulus (CS) in the context of reversal learning, a type of associative learning. The impact of expected uncertainty (the variability of rewards) and unexpected uncertainty (reversal) on the evolving temporal dynamics of the two types of valence representations of the CS was tested in two experiments. Results show that in an environment presenting the two types of uncertainty, the adaptation process (learning rate) of the choices and of the semantic valence representation is slower than the adaptation of the affective valence representation. In contrast, in environments with only unexpected uncertainty (i.e., fixed rewards), there is no difference in the temporal dynamics of the two types of valence representations. Implications for models of affect, value-based learning theories, and value-based decision-making models are discussed.

Distinct reinforcement learning profiles distinguish between language and attentional neurodevelopmental disorders.

BACKGROUND: Theoretical models posit abnormalities in cortico-striatal pathways in two of the most common neurodevelopmental disorders (Developmental dyslexia, DD, and Attention deficit hyperactive disorder, ADHD), but it is still unclear what distinct cortico-striatal dysfunction might distinguish language disorders from others that exhibit very different symptomatology. Although impairments in tasks that depend on the cortico-striatal network, including reinforcement learning (RL), have been implicated in both disorders, there has been little attempt to dissociate between different types of RL or to compare learning processes in these two types of disorders. The present study builds upon prior research indicating the existence of two learning manifestations of RL and evaluates whether these processes can be differentiated in language and attention deficit disorders. We used a two-step RL task shown to dissociate model-based from model-free learning in human learners. RESULTS: Our results show that, relative to neurotypicals, DD individuals showed an impairment in model-free but not in model-based learning, whereas in ADHD the ability to use both model-free and model-based learning strategies was significantly compromised. CONCLUSIONS: Thus, learning impairments in DD may be linked to a selective deficit in the ability to form action-outcome associations based on previous history, whereas in ADHD some learning deficits may be related to an incapacity to pursue rewards based on the tasks' structure. Our results indicate how different patterns of learning deficits may underlie different disorders, and how computation-minded experimental approaches can differentiate between them.

Using clustering algorithms to examine the association between working memory training trajectories and therapeutic outcomes among psychiatric and healthy populations.

Working memory (WM) training has gained interest due to its potential to enhance cognitive functioning and reduce symptoms of mental disorders. Nevertheless, inconsistent results suggest that individual differences may have an impact on training efficacy. This study examined whether individual differences in training performance can predict therapeutic outcomes of WM training, measured as changes in anxiety and depression symptoms in sub-clinical and healthy populations. The study also investigated the association between cognitive abilities at baseline and different training improvement trajectories. Ninety-six participants (50 females, mean age = 27.67, SD = 8.84) were trained using the same WM training task (duration ranged between 7 to 15 sessions). An algorithm was then used to cluster them based on their learning trajectories. We found three main WM training trajectories, which in turn were related to changes in anxiety symptoms following the training. Additionally, executive function abilities at baseline predicted training trajectories. These findings highlight the potential for using clustering algorithms to reveal the benefits of cognitive training to alleviate maladaptive psychological symptoms.

Resting-state heart rate variability (HRV) mediates the association between perceived chronic stress and ambiguity avoidance.

Chronic stress is associated with profound behavioral and physiological alterations, including intolerance to uncertainty and reduced resting-state heart-rate-variability (HRV). Critically, uncertainty may arise in situations with known probabilities (risk) or unknown probabilities (ambiguity). Whether associations between chronic stress and decision-making under uncertainty are dependent on the specific type of uncertain decisions, and whether physiological alterations play a role in these putative associations is not yet clear. Here, ninety-two healthy adults that exhibit various levels of perceived chronic stress underwent resting-state HRV recording before completing a behavioral task that involves decision-making under either risk or ambiguity. Computational modelling quantified participants' behavioral attitudes of approach and avoidance separately for risk and ambiguity. Results indicate, as expected, that perceived chronic stress is positively associated with intolerance to uncertainty and negatively associated with resting-state HRV. Contrary to expectations, behavioral attitudes towards risk and ambiguity were not directly associated with perceived chronic stress, yet HRV fully mediated the association between chronic stress and ambiguity avoidance. Taken together and given the direction of the associations, elevated HRV despite chronic stress may foster adaptive behavior in the form of avoiding ambiguous situations, and hence contribute to reduced exposure to uncertainty and to lower levels of allostatic load.

Cost-benefit considerations have limited effect on the decision to exert cognitive effort in real-world computer-programming tasks.

Recent research shows that people usually try to avoid exerting cognitive effort yet they are willing to exert effort to gain rewards. This cost-benefit framework provides predictions for behaviour outside the laboratory. Nevertheless, the extent to which such considerations affect real-life decisions is not clear. Here we experimentally examined computer-programmers' decisions to write code in a reusable manner, using functions, which demands an initial investment of cognitive effort or to clone and tweak code, a strategy whose cost increases with repetitions. Only a small portion of our participants demonstrated sensitivity to the benefits and costs of programming strategies. When asked to solve the tasks, participants tended to avoid using functions, demonstrating biased effort estimation. By contrast, when asked how they planned to solve the tasks, participants tended to demonstrate an opposite trend, overestimating effort, in line with an injunctive norm involving the overuse of functions. In the context of real-world problems, the effect of cost-benefit considerations may therefore be limited by task-irrelevant factors. Our interdisciplinary approach may be useful in providing novel theoretical insights and in promoting cognitive-effort investments outside the laboratory.

Blend in or stand out: social anxiety levels shape information-sharing strategies.

Although living in social groups provides many benefits for group members, such groups also serve as a setting for social competition over rank and influence. Evolutionary accounts suggest that social anxiety plays a role in regulating in-group conflict, as individuals who are concerned about social threat may choose to defer to others to maintain the hierarchical status quo. Here, we examine how social anxiety levels are related to the advice-giving style an individual adopts: a competitive influence-seeking strategy or a defensive blend-in strategy. We begin by demonstrating that similarity to others drives activity in the brain's valuation system, even during a competitive advice-taking task. Then, in three behavioural experiments, we show that social anxiety levels are related to the tendency to give advice resembling the advice given by rival advisers and to refrain from status-seeking behaviour. Social anxiety was also associated with negative social comparisons with rival advisers. Our findings highlight the role of competing social goals in shaping information sharing.

The devil is in the task structure.

Conflicting evidence about how the brain processes social and individual learning stems from which type of information is presented as the primary source of knowledge during experiments.

Using deep learning to predict human decisions and using cognitive models to explain deep learning models.

Deep neural networks (DNNs) models have the potential to provide new insights in the study of cognitive processes, such as human decision making, due to their high capacity and data-driven design. While these models may be able to go beyond theory-driven models in predicting human behaviour, their opaque nature limits their ability to explain how an operation is carried out, undermining their usefulness as a scientific tool. Here we suggest the use of a DNN model as an exploratory tool to identify predictable and consistent human behaviour, and using explicit, theory-driven models, to characterise the high-capacity model. To demonstrate our approach, we trained an exploratory DNN model to predict human decisions in a four-armed bandit task. We found that this model was more accurate than two explicit models, a reward-oriented model geared towards choosing the most rewarding option, and a reward-oblivious model that was trained to predict human decisions without information about rewards. Using experimental simulations, we were able to characterise the exploratory model using the explicit models. We found that the exploratory model converged with the reward-oriented model's predictions when one option was clearly better than the others, but that it predicted pattern-based explorations akin to the reward-oblivious model's predictions. These results suggest that predictable decision patterns that are not solely reward-oriented may contribute to human decisions. Importantly, we demonstrate how theory-driven cognitive models can be used to characterise the operation of DNNs, making DNNs a useful explanatory tool in scientific investigation.

Adaptive Empathy: A Model for Learning Empathic Responses in Response to Feedback.

Empathy is usually deployed in social interactions. Nevertheless, common measures and examinations of empathy study this construct in isolation from the person in distress. In this article we seek to extend the field of examination to include both empathizer and target to determine whether and how empathic responses are affected by feedback and learned through interaction. Building on computational approaches in feedback-based adaptations (e.g., no feedback, model-free and model-based learning), we propose a framework for understanding how empathic responses are learned on the basis of feedback. In this framework, adaptive empathy, defined as the ability to adapt one's empathic responses, is a central aspect of empathic skills and can provide a new dimension to the evaluation and investigation of empathy. By extending existing neural models of empathy, we suggest that adaptive empathy may be mediated by interactions between the neural circuits associated with valuation, shared distress, observation-execution, and mentalizing. Finally, we propose that adaptive empathy should be considered a prominent facet of empathic capabilities with the potential to explain empathic behavior in health and psychopathology.

To shoot or not to shoot: experiments on moral injury in the context of West Bank checkpoints and COVID-19 restrictions enforcement.

Background: Does exposure to events that transgress accepted norms, such as killing innocent civilians, prompt the psychological and emotional consequences of moral injury among soldiers? Moral injury is associated with negative emotions such as guilt, shame and anger, and a sense of betrayal and is identified among veterans following exposure to potentially morally injurious events (PMIE). Objective: We experimentally investigate how PMIE characteristics affect the intensity of MI and related negative moral emotions in participants with varied military experience. Method: We conducted three controlled, randomized experiments. Each exposed male respondents with active combat experience (Study 1) and varied military experience (Study 2) to four textual vignettes describing PMIE (child/adult and innocent/non-innocent suspect) that transpire at an Israeli checkpoint in the West Bank. In study 3, we exposed participants to two scenarios, where descriptions of police officers enforcing COVID 19 restrictions confronted lockdown violators. Results: Participants assigned to vignettes describing killing an innocent civilian exhibited more intense levels of shame and guilt than those assigned to vignettes describing killing a person carrying a bomb. Religiosity and political ideology were strong predictors of guilt and shame in response to descriptions of checkpoint shootings. These effects disappeared in Study 3, suggesting that political ideology drives MI in intergroup conflict. Conclusions: Background and PMIE-related characteristics affect the development of moral injury. Additionally, lab experiments demonstrate the potential and limitations of controlled studies of moral injury and facilitate an understanding of the aetiology of moral injury in a way unavailable to clinicians. Finally, experimental findings and methodologies offer further insights into the genesis of moral injury and avenues for therapy and prophylaxis.

Antecedentes: ¿La exposición a eventos que transgreden las normas aceptadas, como matar a civiles inocentes, provocan las consecuencias psicológicas y emocionales del daño moral entre los soldados? El daño moral (DM) se asocia con emociones negativas como la culpa, la vergüenza y la ira, y un sentido de traición y es identificado entre los veteranos después de la exposición a eventos potencialmente dañinos moralmente (EPDM).Objetivo: Investigamos experimentalmente cómo las características de EPDM afectan la intensidad del DM y emociones moralmente negativas relacionadas en participantes con vasta experiencia militar.Método: Realizamos tres experimentos controlados y aleatorizados. Cada varón expuesto respondió con experiencia en combate activo (Estudio 1) y vasta experiencia militar (Estudio 2) a cuatro viñetas textuales que describen EPDM (niño/adulto y sospechoso inocente/no inocente) que suceden en un puesto de control israelí en Cisjordania. En el estudio 3, expusimos a los participantes a dos escenarios, donde las descripciones de los agentes de policía que aplicaban las restricciones de COVID-19 enfrentaron a los infractores del confinamiento.Resultados: Los participantes asignados a viñetas que describen el asesinato de un civil inocente exhibieron niveles más intensos de vergüenza y culpa que los asignados a las viñetas que describen el asesinato de una persona llevando una bomba. La religiosidad y la ideología política fueron fuertes predictores de culpa y vergüenza en respuesta a descripciones de tiroteos en puestos de control. Estos efectos desaparecieron en el Estudio 3, lo que sugiere que la ideología política impulsa al DM en los conflictos intergrupales.Conclusiones: Los antecedentes y las características relacionadas con el EPDM afectan el desarrollo del daño moral. Adicionalmente, los experimentos de laboratorio demuestran el potencial y las limitaciones de los estudios de daño moral y facilitan una comprensión de la etiología del daño moral de una manera no disponible para los clínicos. Por último, los hallazgos y las metodologías experimentales ofrecen perspectivas adicionales en la génesis del daño moral y las vías para la terapia y la profilaxis.

The effect of feedback valence and source on perception and metacognition: An fMRI investigation.

Receiving feedback from our environment that informs us about the outcomes of our actions helps us assess our abilities (e.g., metacognition) and to flexibly adapt our behavior, consequently increasing our chances of success. However, a detailed examination of the effect of feedback on the brain activation during perceptual and confidence judgments as well as the interrelations between perceptual accuracy, prospective and retrospective confidence remains unclear. Here we used functional magnetic resonance imaging (fMRI) to examine the neural response to feedback valence and source in visual contrast discrimination together with prospective confidence judgments at the beginning of each block and retrospective confidence judgments after every decision. Positive feedback was associated with higher activation (or lower deactivation depending on the area) in areas previously involved in attention, performance monitoring and visual regions during the perceptual judgment than during the confidence judgment. Changes in prospective confidence were positively related to changes in perceptual accuracy as well as to the corresponding retrospective confidence. Thus, feedback information impacted multiple, qualitatively different brain processing states, and we also revealed the dynamic interplay between prospective, perceptual accuracy and retrospective self-assessment.

Automatically controlled: Task irrelevance fully cancels otherwise automatic imitation.

Automatic imitation refers to the act of unintentionally mimicking observed actions. Inspired by a theoretical framework that allows for controlled yet unintentional processes, we tested whether automatic imitation depends on the task relevance of the to-be-imitated movements. Replicating previous results, we find that movements that are part of the participant's task set unintentionally influence response. Our key finding is that participants generally do not imitate similar and highly familiar movements that are not part of the task set and hence are task-irrelevant. Furthermore, the results of computational data modeling are consistent with the notion that task-relevance modulates the mental activation of information, as posited by the above theoretical framework. Our findings are not predicted and cannot be explained using current accounts of automatic imitation, such as Associative Sequence Learning or Theory of Event Coding. At a broader level, the key contribution of this study is in challenging the empirical basis for automatic imitation by showing that the effects interpreted as imitation occur only for task relevant responses. This pattern lends itself to a different interpretation which is not related to imitation, automatic or otherwise, but rather to the general phenomenon of response compatibility effects. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Strategic disinformation outperforms honesty in competition for social influence.

Competition for social influence is a major force shaping societies, from baboons guiding their troop in different directions, to politicians competing for voters, to influencers competing for attention on social media. Social influence is invariably a competitive exercise with multiple influencers competing for it. We study which strategy maximizes social influence under competition. Applying game theory to a scenario where two advisers compete for the attention of a client, we find that the rational solution for advisers is to communicate truthfully when favored by the client, but to lie when ignored. Across seven pre-registered studies, testing 802 participants, such a strategic adviser consistently outcompeted an honest adviser. Strategic dishonesty outperformed truth-telling in swaying individual voters, the majority vote in anonymously voting groups, and the consensus vote in communicating groups. Our findings help explain the success of political movements that thrive on disinformation, and vocal underdog politicians with no credible program.

Trusting and learning from others: immediate and long-term effects of learning from observation and advice.

Social learning underpins our species's extraordinary success. Learning through observation has been investigated in several species, but learning from advice-where information is intentionally broadcast-is less understood. We used a pre-registered, online experiment (n = 1492) combined with computational modelling to examine learning through observation and advice. Participants were more likely to immediately follow advice than to copy an observed choice, but this was dependent upon trust in the adviser: highly paranoid participants were less likely to follow advice in the short term. Reinforcement learning modelling revealed two distinct patterns regarding the long-term effects of social information: some individuals relied fully on social information, whereas others reverted to trial-and-error learning. This variation may affect the prevalence and fidelity of socially transmitted information. Our results highlight the privileged status of advice relative to observation and how the assimilation of intentionally broadcast information is affected by trust in others.

Adaptive Empathy: Empathic Response Selection as a Dynamic, Feedback-Based Learning Process.

Empathy allows us to respond to the emotional state of another person. Considering that an empathic interaction may last beyond the initial response, learning mechanisms may be involved in dynamic adaptation of the reaction to the changing emotional state of the other person. However, traditionally, empathy is assessed through sets of isolated reactions to another's distress. Here we address this gap by focusing on adaptive empathy, defined as the ability to learn and adjust one's empathic responses based on feedback. For this purpose, we designed a novel paradigm of associative learning in which participants chose one of two empathic strategies (reappraisal or distraction) to attenuate the distress of a target person, where one strategy had a higher probability of relieving distress. After each choice, participants received feedback about the success of their chosen strategy in relieving the target person's distress, which they could use to inform their future decisions. The results show that the participants made more accurate choices in the adaptive empathy condition than in a non-social control condition, pointing to an advantage for learning from social feedback. We found a correlation between adaptive empathy and a trait measure of cognitive empathy. These findings indicate that the ability to learn about the effectiveness of empathic responses may benefit from incorporating mentalizing abilities. Our findings provide a lab-based model for studying adaptive empathy and point to the potential contribution of learning theory to enhancing our understanding of the dynamic nature of empathy.

Learning how to behave: cognitive learning processes account for asymmetries in adaptation to social norms.

Changes to social settings caused by migration, cultural change or pandemics force us to adapt to new social norms. Social norms provide groups of individuals with behavioural prescriptions and therefore can be inferred by observing their behaviour. This work aims to examine how cognitive learning processes affect adaptation and learning of new social norms. Using a multiplayer game, I found that participants initially complied with various social norms exhibited by the behaviour of bot-players. After gaining experience with one norm, adaptation to a new norm was observed in all cases but one, where an active-harm norm was resistant to adaptation. Using computational learning models, I found that active behaviours were learned faster than omissions, and harmful behaviours were more readily attributed to all group members than beneficial behaviours. These results provide a cognitive foundation for learning and adaptation to descriptive norms and can inform future investigations of group-level learning and cross-cultural adaptation.