Beyond the matrix: Experimental approaches to studying cognitive agents in social-ecological systems.

Social-ecological systems, in which agents interact with each other and their environment are important both for sustainability applications and for under- standing how human cognition functions in context. In such systems, the en- vironment shapes the agents' experience and actions, and in turn collective action of agents changes social and physical aspects of the environment. Here we review current investigation approaches, which rely on a lean design, with discrete actions and outcomes and little scope for varying environmental pa- rameters and cognitive demands. We then introduce multiagent reinforcement learning (MARL) approach, which builds on modern artificial intelligence tech- niques, which provides new avenues to model complex social worlds, while pre- serving more of their characteristics, and allowing them to capture a variety of social phenomena. These techniques can be fed back to the laboratory where they make it easier to design experiments in complex social situations without compromising their tractability for computational modeling. We showcase the potential MARL by discussing several recent studies that have used it, detail- ing the way environmental settings and cognitive constraints can lead to the emergence of complex cooperation strategies. This novel approach can help re- searchers bring together insights from human cognition, sustainability, and AI, to tackle real world problems of social-ecological systems.

A cognitive approach to learning, monitoring, and shifting social norms.

Social norms govern and prescribe how group members behave. Since norms manifest in individuals' behavior, it is important to consider the cognitive demands associated with detecting and monitoring norm behaviors. Here I describe three types of norms that differ in the behavior they prescribe, the cognitive processes of behavior detection and monitoring they require, and the compliance and cooperative patterns they entail. Categorical norms, such as taboos, prescribe what actions group members must or shouldn't do, and may rely on affective outcomes. Scale-sensitive norms govern how much of a behavior one must do and rely on signal detection processes. History-sensitive norms consider a whole sequence of actions performed by specific individuals, such as the history of contribution, and require evidence accumulation mechanisms. Detecting and monitoring these different types of behaviors poses different cognitive demands, which may affect the extent and stability of social norms. By taking the cognitive perspective, it may be possible to understand why some norms are more resilient than others, and plan interventions that promote norm change by matching these cognitive demands.

Asymmetric cognitive learning mechanisms underlying the persistence of intergroup bias.

Intergroup bias, the tendency to favor ingroups and be hostile towards outgroups, underlies many societal problems and persists even when intergroup members interact and share experiences. Here we study the way cognitive learning processes contribute to the persistence of intergroup bias. Participants played a game with ingroup and outgroup bot-players that entailed collecting stars and could sacrifice a move to zap another player. We found that intergroup bias persisted as participants were more likely to zap outgroup players, regardless of their zapping behavior. Using a computational model, we found that this bias was caused by asymmetries in three learning mechanisms. Participants had a greater prior bias to zap out-group players, they learned more readily about the negative behavior of out-groups and were less likely to attribute the positive behavior of one out-group player to other out-group players. Our results uncover the way cognitive social learning mechanisms shape and confound intergroup dynamics.

Changing attachment orientation: Uncovering the role of shifting the emotion regulation tendency.

Changes in the individual's attachment orientation toward greater security are considered an important clinical goal. One promising underlying process of change in attachment orientation is shifting the emotion regulation tendency, in which the individual progresses from overreliance on the self or on the other to regulate emotional arousal. The present study utilized a computational approach to study shifts in the emotion regulation tendency as these manifest in the patient's and therapist's vocally encoded emotional arousal. The study examined whether shifts in the regulation tendency are associated with decreases in the level of insecure attachment and in strengthening of the therapeutic alliance. Shifts in the regulation tendency were examined throughout the early stages of treatment (Sessions 1-4) using 11,710 talk turns within 52 patient-therapist dyads. Findings suggest that shifts in the emotion regulation tendency are associated with greater strengthening of the therapeutic alliance and a decrease in the level of attachment avoidance. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Comparing experience- and description-based economic preferences across 11 countries.

Recent evidence indicates that reward value encoding in humans is highly context dependent, leading to suboptimal decisions in some cases, but whether this computational constraint on valuation is a shared feature of human cognition remains unknown. Here we studied the behaviour of n = 561 individuals from 11 countries of markedly different socioeconomic and cultural makeup. Our findings show that context sensitivity was present in all 11 countries. Suboptimal decisions generated by context manipulation were not explained by risk aversion, as estimated through a separate description-based choice task (that is, lotteries) consisting of matched decision offers. Conversely, risk aversion significantly differed across countries. Overall, our findings suggest that context-dependent reward value encoding is a feature of human cognition that remains consistently present across different countries, as opposed to description-based decision-making, which is more permeable to cultural factors.

Disentangling the contribution of individual and social learning processes in human advice-taking behavior.

The study of social learning examines how individuals learn from others by means of observation, imitation, or compliance with advice. However, it still remains largely unknown whether social learning processes have a distinct contribution to behavior, independent from non-social trial-and-error learning that often occurs simultaneously. 153 participants completed a reinforcement learning task, where they were asked to make choices to gain rewards. Advice from an artificial teacher was presented in 60% of the trials, allowing us to compare choice behavior with and without advice. Results showed a strong and reliable tendency to follow advice (test-retest reliability ~0.73). Computational modeling suggested a unique contribution of three distinct learning strategies: (a) individual learning (i.e., learning the value of actions, independent of advice), (b) informed advice-taking (i.e., learning the value of following advice), and (c) non-informed advice-taking (i.e., a constant bias to follow advice regardless of outcome history). Comparing artificial and empirical data provided specific behavioral regression signatures to both informed and non-informed advice taking processes. We discuss the theoretical implications of integrating internal and external information during the learning process.

The spread of affective and semantic valence representations across states.

In many decision problems, outcomes are not reached after a single action but rather after a series of events or states. To optimize decisions over multiple states, representations of how good or bad the outcomes are, that is, the outcomes' valence, should spread across states. One mechanism for valence spreading is a temporal, state-independent process in which a single valence representation is updated when an outcome is experienced and fades away afterwards. Each state's valence is based on its temporal proximity to the experienced outcome. An alternative, state-dependent mechanism relies on the structure of transitions between states, updating a separate valence representation for each state according to its spatial distance from the outcomes. We examined how these mechanistic accounts shape the spread of two formats of valence representation, feelings (affective valence) and knowledge (semantic valence), between states. In two pre-registered experiments (N = 585), we used a novel task in which participants move in a four-state maze, one of which contains an outcome. The participants provide self-reports of affective and semantic valence throughout the maze and after finishing it. Results show that the affective representation of negative valence is more localized in state-space than the semantic representation. We also found evidence for the relative reliance of the affective valence on a temporal, state-independent mechanism and of the semantic valence on a structured, state-dependent mechanism. Our findings provide mechanistic accounts for the differences between affective and semantic valence representations and indicate how such representations may play a role in associative learning and decision-making.

Outcome context-dependence is not WEIRD: Comparing reinforcement- and description-based economic preferences worldwide.

Recent evidence indicates that reward value encoding in humans is highly context-dependent, leading to suboptimal decisions in some cases. But whether this computational constraint on valuation is a shared feature of human cognition remains unknown. To address this question, we studied the behavior of individuals from across 11 countries of markedly different socioeconomic and cultural makeup using an experimental approach that reliably captures context effects in reinforcement learning. Our findings show that all samples presented evidence of similar sensitivity to context. Crucially, suboptimal decisions generated by context manipulation were not explained by risk aversion, as estimated through a separate description-based choice task (i.e., lotteries) consisting of matched decision offers. Conversely, risk aversion significantly differed across countries. Overall, our findings suggest that context-dependent reward value encoding is a hardcoded feature of human cognition, while description-based decision-making is significantly sensitive to cultural factors.

Distinct reinforcement learning profiles distinguish between language and attentional neurodevelopmental disorders.

BACKGROUND: Theoretical models posit abnormalities in cortico-striatal pathways in two of the most common neurodevelopmental disorders (Developmental dyslexia, DD, and Attention deficit hyperactive disorder, ADHD), but it is still unclear what distinct cortico-striatal dysfunction might distinguish language disorders from others that exhibit very different symptomatology. Although impairments in tasks that depend on the cortico-striatal network, including reinforcement learning (RL), have been implicated in both disorders, there has been little attempt to dissociate between different types of RL or to compare learning processes in these two types of disorders. The present study builds upon prior research indicating the existence of two learning manifestations of RL and evaluates whether these processes can be differentiated in language and attention deficit disorders. We used a two-step RL task shown to dissociate model-based from model-free learning in human learners. RESULTS: Our results show that, relative to neurotypicals, DD individuals showed an impairment in model-free but not in model-based learning, whereas in ADHD the ability to use both model-free and model-based learning strategies was significantly compromised. CONCLUSIONS: Thus, learning impairments in DD may be linked to a selective deficit in the ability to form action-outcome associations based on previous history, whereas in ADHD some learning deficits may be related to an incapacity to pursue rewards based on the tasks' structure. Our results indicate how different patterns of learning deficits may underlie different disorders, and how computation-minded experimental approaches can differentiate between them.

Temporal dynamics of the semantic versus affective representations of valence during reversal learning.

Valence, the representation of a stimulus in terms of good or bad, plays a central role in models of affect, value-based learning theories, and value-based decision-making models. Previous work used Unconditioned Stimulus (US) to support a theoretical division between two different types of valence representations for a stimulus: the semantic representation of valence, i.e., stored accumulated knowledge about the value of the stimulus, and the affective representation of valence, i.e., the valence of the affective response to this stimulus. The current work extended past research by using a neutral Conditioned Stimulus (CS) in the context of reversal learning, a type of associative learning. The impact of expected uncertainty (the variability of rewards) and unexpected uncertainty (reversal) on the evolving temporal dynamics of the two types of valence representations of the CS was tested in two experiments. Results show that in an environment presenting the two types of uncertainty, the adaptation process (learning rate) of the choices and of the semantic valence representation is slower than the adaptation of the affective valence representation. In contrast, in environments with only unexpected uncertainty (i.e., fixed rewards), there is no difference in the temporal dynamics of the two types of valence representations. Implications for models of affect, value-based learning theories, and value-based decision-making models are discussed.

Using clustering algorithms to examine the association between working memory training trajectories and therapeutic outcomes among psychiatric and healthy populations.

Working memory (WM) training has gained interest due to its potential to enhance cognitive functioning and reduce symptoms of mental disorders. Nevertheless, inconsistent results suggest that individual differences may have an impact on training efficacy. This study examined whether individual differences in training performance can predict therapeutic outcomes of WM training, measured as changes in anxiety and depression symptoms in sub-clinical and healthy populations. The study also investigated the association between cognitive abilities at baseline and different training improvement trajectories. Ninety-six participants (50 females, mean age = 27.67, SD = 8.84) were trained using the same WM training task (duration ranged between 7 to 15 sessions). An algorithm was then used to cluster them based on their learning trajectories. We found three main WM training trajectories, which in turn were related to changes in anxiety symptoms following the training. Additionally, executive function abilities at baseline predicted training trajectories. These findings highlight the potential for using clustering algorithms to reveal the benefits of cognitive training to alleviate maladaptive psychological symptoms.

Resting-state heart rate variability (HRV) mediates the association between perceived chronic stress and ambiguity avoidance.

Chronic stress is associated with profound behavioral and physiological alterations, including intolerance to uncertainty and reduced resting-state heart-rate-variability (HRV). Critically, uncertainty may arise in situations with known probabilities (risk) or unknown probabilities (ambiguity). Whether associations between chronic stress and decision-making under uncertainty are dependent on the specific type of uncertain decisions, and whether physiological alterations play a role in these putative associations is not yet clear. Here, ninety-two healthy adults that exhibit various levels of perceived chronic stress underwent resting-state HRV recording before completing a behavioral task that involves decision-making under either risk or ambiguity. Computational modelling quantified participants' behavioral attitudes of approach and avoidance separately for risk and ambiguity. Results indicate, as expected, that perceived chronic stress is positively associated with intolerance to uncertainty and negatively associated with resting-state HRV. Contrary to expectations, behavioral attitudes towards risk and ambiguity were not directly associated with perceived chronic stress, yet HRV fully mediated the association between chronic stress and ambiguity avoidance. Taken together and given the direction of the associations, elevated HRV despite chronic stress may foster adaptive behavior in the form of avoiding ambiguous situations, and hence contribute to reduced exposure to uncertainty and to lower levels of allostatic load.

Cost-benefit considerations have limited effect on the decision to exert cognitive effort in real-world computer-programming tasks.

Recent research shows that people usually try to avoid exerting cognitive effort yet they are willing to exert effort to gain rewards. This cost-benefit framework provides predictions for behaviour outside the laboratory. Nevertheless, the extent to which such considerations affect real-life decisions is not clear. Here we experimentally examined computer-programmers' decisions to write code in a reusable manner, using functions, which demands an initial investment of cognitive effort or to clone and tweak code, a strategy whose cost increases with repetitions. Only a small portion of our participants demonstrated sensitivity to the benefits and costs of programming strategies. When asked to solve the tasks, participants tended to avoid using functions, demonstrating biased effort estimation. By contrast, when asked how they planned to solve the tasks, participants tended to demonstrate an opposite trend, overestimating effort, in line with an injunctive norm involving the overuse of functions. In the context of real-world problems, the effect of cost-benefit considerations may therefore be limited by task-irrelevant factors. Our interdisciplinary approach may be useful in providing novel theoretical insights and in promoting cognitive-effort investments outside the laboratory.

Blend in or stand out: social anxiety levels shape information-sharing strategies.

Although living in social groups provides many benefits for group members, such groups also serve as a setting for social competition over rank and influence. Evolutionary accounts suggest that social anxiety plays a role in regulating in-group conflict, as individuals who are concerned about social threat may choose to defer to others to maintain the hierarchical status quo. Here, we examine how social anxiety levels are related to the advice-giving style an individual adopts: a competitive influence-seeking strategy or a defensive blend-in strategy. We begin by demonstrating that similarity to others drives activity in the brain's valuation system, even during a competitive advice-taking task. Then, in three behavioural experiments, we show that social anxiety levels are related to the tendency to give advice resembling the advice given by rival advisers and to refrain from status-seeking behaviour. Social anxiety was also associated with negative social comparisons with rival advisers. Our findings highlight the role of competing social goals in shaping information sharing.

The devil is in the task structure.

Conflicting evidence about how the brain processes social and individual learning stems from which type of information is presented as the primary source of knowledge during experiments.

Using deep learning to predict human decisions and using cognitive models to explain deep learning models.

Deep neural networks (DNNs) models have the potential to provide new insights in the study of cognitive processes, such as human decision making, due to their high capacity and data-driven design. While these models may be able to go beyond theory-driven models in predicting human behaviour, their opaque nature limits their ability to explain how an operation is carried out, undermining their usefulness as a scientific tool. Here we suggest the use of a DNN model as an exploratory tool to identify predictable and consistent human behaviour, and using explicit, theory-driven models, to characterise the high-capacity model. To demonstrate our approach, we trained an exploratory DNN model to predict human decisions in a four-armed bandit task. We found that this model was more accurate than two explicit models, a reward-oriented model geared towards choosing the most rewarding option, and a reward-oblivious model that was trained to predict human decisions without information about rewards. Using experimental simulations, we were able to characterise the exploratory model using the explicit models. We found that the exploratory model converged with the reward-oriented model's predictions when one option was clearly better than the others, but that it predicted pattern-based explorations akin to the reward-oblivious model's predictions. These results suggest that predictable decision patterns that are not solely reward-oriented may contribute to human decisions. Importantly, we demonstrate how theory-driven cognitive models can be used to characterise the operation of DNNs, making DNNs a useful explanatory tool in scientific investigation.

To shoot or not to shoot: experiments on moral injury in the context of West Bank checkpoints and COVID-19 restrictions enforcement.

Background: Does exposure to events that transgress accepted norms, such as killing innocent civilians, prompt the psychological and emotional consequences of moral injury among soldiers? Moral injury is associated with negative emotions such as guilt, shame and anger, and a sense of betrayal and is identified among veterans following exposure to potentially morally injurious events (PMIE). Objective: We experimentally investigate how PMIE characteristics affect the intensity of MI and related negative moral emotions in participants with varied military experience. Method: We conducted three controlled, randomized experiments. Each exposed male respondents with active combat experience (Study 1) and varied military experience (Study 2) to four textual vignettes describing PMIE (child/adult and innocent/non-innocent suspect) that transpire at an Israeli checkpoint in the West Bank. In study 3, we exposed participants to two scenarios, where descriptions of police officers enforcing COVID 19 restrictions confronted lockdown violators. Results: Participants assigned to vignettes describing killing an innocent civilian exhibited more intense levels of shame and guilt than those assigned to vignettes describing killing a person carrying a bomb. Religiosity and political ideology were strong predictors of guilt and shame in response to descriptions of checkpoint shootings. These effects disappeared in Study 3, suggesting that political ideology drives MI in intergroup conflict. Conclusions: Background and PMIE-related characteristics affect the development of moral injury. Additionally, lab experiments demonstrate the potential and limitations of controlled studies of moral injury and facilitate an understanding of the aetiology of moral injury in a way unavailable to clinicians. Finally, experimental findings and methodologies offer further insights into the genesis of moral injury and avenues for therapy and prophylaxis.

Antecedentes: ¿La exposición a eventos que transgreden las normas aceptadas, como matar a civiles inocentes, provocan las consecuencias psicológicas y emocionales del daño moral entre los soldados? El daño moral (DM) se asocia con emociones negativas como la culpa, la vergüenza y la ira, y un sentido de traición y es identificado entre los veteranos después de la exposición a eventos potencialmente dañinos moralmente (EPDM).Objetivo: Investigamos experimentalmente cómo las características de EPDM afectan la intensidad del DM y emociones moralmente negativas relacionadas en participantes con vasta experiencia militar.Método: Realizamos tres experimentos controlados y aleatorizados. Cada varón expuesto respondió con experiencia en combate activo (Estudio 1) y vasta experiencia militar (Estudio 2) a cuatro viñetas textuales que describen EPDM (niño/adulto y sospechoso inocente/no inocente) que suceden en un puesto de control israelí en Cisjordania. En el estudio 3, expusimos a los participantes a dos escenarios, donde las descripciones de los agentes de policía que aplicaban las restricciones de COVID-19 enfrentaron a los infractores del confinamiento.Resultados: Los participantes asignados a viñetas que describen el asesinato de un civil inocente exhibieron niveles más intensos de vergüenza y culpa que los asignados a las viñetas que describen el asesinato de una persona llevando una bomba. La religiosidad y la ideología política fueron fuertes predictores de culpa y vergüenza en respuesta a descripciones de tiroteos en puestos de control. Estos efectos desaparecieron en el Estudio 3, lo que sugiere que la ideología política impulsa al DM en los conflictos intergrupales.Conclusiones: Los antecedentes y las características relacionadas con el EPDM afectan el desarrollo del daño moral. Adicionalmente, los experimentos de laboratorio demuestran el potencial y las limitaciones de los estudios de daño moral y facilitan una comprensión de la etiología del daño moral de una manera no disponible para los clínicos. Por último, los hallazgos y las metodologías experimentales ofrecen perspectivas adicionales en la génesis del daño moral y las vías para la terapia y la profilaxis.

Adaptive Empathy: A Model for Learning Empathic Responses in Response to Feedback.

Empathy is usually deployed in social interactions. Nevertheless, common measures and examinations of empathy study this construct in isolation from the person in distress. In this article we seek to extend the field of examination to include both empathizer and target to determine whether and how empathic responses are affected by feedback and learned through interaction. Building on computational approaches in feedback-based adaptations (e.g., no feedback, model-free and model-based learning), we propose a framework for understanding how empathic responses are learned on the basis of feedback. In this framework, adaptive empathy, defined as the ability to adapt one's empathic responses, is a central aspect of empathic skills and can provide a new dimension to the evaluation and investigation of empathy. By extending existing neural models of empathy, we suggest that adaptive empathy may be mediated by interactions between the neural circuits associated with valuation, shared distress, observation-execution, and mentalizing. Finally, we propose that adaptive empathy should be considered a prominent facet of empathic capabilities with the potential to explain empathic behavior in health and psychopathology.