Exploring the limits of hierarchical world models in reinforcement learning.

Hierarchical model-based reinforcement learning (HMBRL) aims to combine the sample efficiency of model-based reinforcement learning with the abstraction capability of hierarchical reinforcement learning. While HMBRL has great potential, the structural and conceptual complexities of current approaches make it challenging to extract general principles, hindering understanding and adaptation to new use cases, and thereby impeding the overall progress of the field. In this work we describe a novel HMBRL framework and evaluate it thoroughly. We construct hierarchical world models that simulate the environment at various levels of temporal abstraction. These models are used to train a stack of agents that communicate top-down by proposing goals to their subordinate agents. A significant focus of this study is the exploration of a static and environment agnostic temporal abstraction, which allows concurrent training of models and agents throughout the hierarchy. Unlike most goal-conditioned H(MB)RL approaches, it also leads to comparatively low dimensional abstract actions. Although our HMBRL approach did not outperform traditional methods in terms of final episode returns, it successfully facilitated decision-making across two levels of abstraction. A central challenge in enhancing our method's performance, as uncovered through comprehensive experimentation, is model exploitation on the abstract level of our world model stack. We provide an in depth examination of this issue, discussing its implications and suggesting directions for future research to overcome this challenge. By sharing these findings, we aim to contribute to the broader discourse on refining HMBRL methodologies.

Predicting technostress: The Big Five model of personality and subjective well-being.

The main goal of the current study is to broaden the knowledge on the association between personality, subjective well-being (SWB) and technostress in an academic context. This research specifically examines the prevalence of technostress in a European university sample. It also explores the relationship between technostress and its dimensions with the Big Five model of personality and with SWB and its affective and cognitive components. Finally, the combined predictive validity of the Big Five and SWB on technostress is tested. The sample was composed of 346 undergraduate students. Correlational and multiple regression analyses were carried out. Results show that fatigue and anxiety are the most frequently experienced dimensions of technostress. Emotional stability, openness to experience, and SWB are negatively and significantly correlated to technostress. Multiple regression analyses show that the Big Five factors and SWB account for technostress variance, the main predictor being the affective component of SWB. These results contribute to a more comprehensive understanding of technostress and suggest that personality traits and SWB are important factors in its prediction. The theoretical and practical implications will be discussed.

Using diffusion models for symbolic numeracy tasks to examine aging effects.

We present a model-based analysis of aging effects in three symbolic numeracy tasks using three groups of subjects (young adults, 60- to 69-year-olds, and 70- to 90-year-olds). The tasks are number discrimination (is this number greater or less than 50), number memory (was this number in the list of numbers just displayed), and number line (point to where this number is on this number line). The first two tasks were fit by the standard two-choice diffusion model and the last one by the spatially continuous diffusion model (Ratcliff, 2018). Results showed good fits of the models to accuracy (choices) and response time distributions. In the tasks, nondecision time (the time to encode a stimulus and make a response) increased with age, but the amount of evidence needed for a decision (boundary settings) increased in the number discrimination and number memory tasks, but not the number line task. The number discrimination task produced conflicting accuracy and response time results as a function of age, but the model-based analyses resolved these differences. In the number memory task, drift rates (evidence used to drive the decision process) were lower for the older adults than for young adults, but for the other two (easier) tasks, there was no change in drift rate with age. The analyses extracted differences among individuals in model components, some of which were systematic across tasks. In particular, drift rates were correlated across tasks, which shows consistent individual differences across tasks, results that could not have been obtained without model-based analyses. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The Role of Diagnostic Models in Prejudice Toward People With Borderline Personality Disorder: An Experimental Investigation.

We tested whether dimensional personality disorder models such as the Alternative Model for Personality Disorders (AMPD) cause less prejudice toward people with borderline personality disorder (BPD) than categorical models, and we sought to identify the mechanisms underpinning this reduction in prejudice. Undergraduate psychology students (n = 183) were randomly assigned to one of three conditions (AMPD, categorical, control) and given descriptive information about BPD. Participants in the AMPD and categorical conditions also received a presentation about their respective BPD diagnostic criteria. Students in all conditions then completed a survey assessing their prejudice toward people with BPD. There was no difference between conditions on overall prejudice toward people with BPD. However, the AMPD increased continuum beliefs and decreased categorical beliefs, and these, in turn, affected perceptions of people with BPD as ingroup members, which indirectly reduced prejudice. We have identified pathways through which the AMPD indirectly reduces prejudice toward people with BPD.

A modified transactional model of stress and coping on depressive symptoms among informal caregivers of persons with dementia.

Informal caregivers are crucial to the care of persons with dementia (PWD), but their role is often reported as challenging. This study aims to examine caregiver stress and depressive symptoms among informal caregivers of PWD using a modified transactional stress and coping model. Path analysis was conducted among a sample of informal caregivers of PWD in Singapore (n = 281) using information on functional dependence and memory and behavioral problems of PWD, and self-reported measurements on caregivers' knowledge of dementia, perceived positive aspects of caregiving, social support, coping patterns, caregiving self-efficacy, caregiving burden and depressive symptoms. The initial model reflecting our assumptions for the transactional stress and coping model showed a poor fit (model 1 - CFI = 0.858, TLI = 0.665, RMSEA = 0.118). Logical modifications were made until sufficient model fit was achieved (model 2 - CFI = 0.987, TLI = 0.955, RMSEA = 0.043). We then removed the insignificant paths in model 2 and obtained our final model (model 3 - CFI = 0.990, TLI = 0.974, RMSEA = 0.033). The final model supported our hypotheses, with some adjustments. This study advances our understanding of caregiver distress by modifying the transactional stress and coping model, including (1) the key role of caregiver self-efficacy in the primary appraisal process, (2) the dynamic assessment of coping resources across all stages of the model, and (3) the importance of coping patterns. Future studies could explore the generalizability of these findings.

ECO-MIND: enhancing pro-environmental behaviours and mental health through nature contact for urban youth - a research protocol for a multi-country study using geographic ecological momentary assessment and mental models.

INTRODUCTION: Urban youth are experiencing increasing mental health problems due to diverse personal, social and environmental concerns. Youths' detachment from natural environments, including green and blue spaces, may intensify such issues further. Contact with nature can benefit mental health and promote pro-environmental behaviour (PEB). Yet, only a few studies assess these relationships among the youth usually ignoring effects of living in diverse urban contexts, and everyday nature experiences. ECO-MIND will investigate whether urban youth's dynamic greenspace exposure and their mental models about nature connectedness explain the associations between greenspace exposure, mental health and PEB in multiple urban contexts. METHODS AND ANALYSIS: We will collect data from university students from the Global South (ie, Dhaka, Kampala) and Global North cities (ie, Utrecht). Participants aged 18-24 will be recruited through stratified random sampling. We will use geographic ecological momentary assessment to assess respondents' everyday experiences and exposure to greenspaces. Our definition of greenspace exposure will be based on the availability, accessibility and visibility of greenspaces extracted from satellite and street view images. We will administer a baseline questionnaire to participants about mental health, nature connectedness and PEB characteristics. Further, we will ask participants to build mental models to show their perception of nature connectedness. Finally, we will fit partial least square structural equation models and multi-level models to our longitudinal geographic momentary assessment data. ETHICS AND DISSEMINATION: The study was approved by the Ethical Review Board of the Utrecht University (Geo S-23221). Informed consent must be given freely, without coercion and based on a clear understanding of the participation in the study. Findings will be disseminated in peer-reviewed journals and at conferences. Furthermore, we will implement public engagement activities (eg, panel discussion) to share results among local stakeholders and policymakers and cocreate policy briefs.

Parent-child communication and educational anxiety: a longitudinal analysis based on the common fate model.

BACKGROUND: In the face of family stress and emerging family problems, the transactional model of stress and coping provides new perspectives for solving stress problems in the family. This study integrates it with the common fate model to reveal new paths for coping with educational anxiety problems from the level of parent-child dichotomous interaction in the family environment, aiming to provide some theoretical and practical support for alleviating parents' education anxiety. METHODS: This study used a combination of interviews and questionnaires to stratify students and their parents from a public junior high school in Jinan City to participate in the study. Data were collected in two time periods (T1, T2; one year apart), and the demographic characteristics of parents and children, parent-child communication quality, education anxiety, and parent-child trust relationship of 495 families were finally analysed by SPSS 27.0. RESULTS: The results found that (1) based on the Common Fate Model, quality of parent-child communication negatively predicted parents' education anxiety; (2) quality of parent-child communication positively predicted parent-child trust relationship; (3) parent-child trust relationship negatively predicted parents'education anxiety; (4) parent-child trust relationship mediated quality of parent-child communication and education anxiety. CONCLUSIONS: This study sheds light on research related to stress coping and anxiety relief in the family environment, and provides theoretical and practical support for understanding the interaction between parents and children at the family level, and coping with stressful events together. It implies that future researchers should not only focus on one aspect when coping with and dealing with family problems, but should analyse the family as a whole, including the communication status and parent-child relationship between parents and children.

Evaluation of an Algorithmic-Level Left-Corner Parsing Account of Surprisal Effects.

This article evaluates the predictions of an algorithmic-level distributed associative memory model as it introduces, propagates, and resolves ambiguity, and compares it to the predictions of computational-level parallel parsing models in which ambiguous analyses are accounted separately in discrete distributions. By superposing activation patterns that serve as cues to other activation patterns, the model is able to maintain multiple syntactically complex analyses superposed in a finite working memory, propagate this ambiguity through multiple intervening words, then resolve this ambiguity in a way that produces a measurable predictor that is proportional to the log conditional probability of the disambiguating word given its context, marginalizing over all remaining analyses. The results are indeed consistent in cases of complex structural ambiguity with computational-level parallel parsing models producing this same probability as a predictor, which have been shown reliably to predict human reading times.

A Simple Computational Model of Semantic Priming in 18-Month-Olds.

We propose a simple computational model that describes potential mechanisms underlying the organization and development of the lexical-semantic system in 18-month-old infants. We focus on two independent aspects: (i) on potential mechanisms underlying the development of taxonomic and associative priming, and (ii) on potential mechanisms underlying the effect of Inter Stimulus Interval on these priming effects. Our model explains taxonomic priming between words by semantic feature overlap, whereas associative priming between words is explained by Hebbian links between semantic representations derived from co-occurrence relations between words (or their referents). From a developmental perspective, any delay in the emergence of taxonomic priming compared to associative priming during infancy seems paradoxical since feature overlap per se need not be learned. We address this paradox in the model by showing that feature overlap itself is an emergent process. The model successfully replicates infant data related to Inter Stimulus Interval effects in priming experiments and makes testable predictions.

Extending a rational process model of causal reasoning: Assessing Markov violations and explaining away with inhibitory causal relations.

Causes sometimes decrease rather increase the probability of an effect, as when drinking coffee lowers the probability of sleep or an aspirin eliminates a headache. This research tests how two causal reasoning errors that have influenced the development of theories of human causal reasoning manifest themselves in the presence of inhibitory causal relations. Past research with generative causal relations (a cause makes its effect more probable) has shown that people violate the Markov condition, the pattern of independence that should obtain among causally related variables. And it has shown that they explain away-the phenomenon in which one should lower likelihood of one event when another is discovered to have occurred (e.g., exonerating one murder suspect when evidence against another is found)-too little or not at all. The new empirical findings reported here reveal that both sorts of errors manifest themselves when inhibitory causal relations are present although, unexpectedly, the direction of those errors sometimes reverses. Only the mutation sampler, a rational process model of human causal reasoning, correctly predicted these novel empirical findings. These results support the view that causal reasoning errors can be understood as arising from rational inference constrained by limited cognitive resources. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Mistakes in Thinking about Cognitive Science and How to Reduce Them.

This article allows readers to assess their ability to detect errors in thinking in seven case histories of psychologists' thoughts about cognitive science. It explains the nature of the errors and shows that some of them reflect faulty reasoning. It presents a "model method" to improve reasoning. It is based on the theory of mental models, which gives a general account of how individuals think, both deductively and indicatively, and which postulates that individuals construct mental models of possibilities in the world. The model method enhances both the accuracy and speed of reasoning. The article concludes with some general reflections on the role of knowledge of meanings, the world, and context in thinking.

On aggregation invariance of multinomial processing tree models.

Multinomial processing tree (MPT) models are prominent and frequently used tools to model and measure cognitive processes underlying responses in many experimental paradigms. Although MPT models typically refer to cognitive processes within single individuals, they have often been applied to group data aggregated across individuals. We investigate the conditions under which MPT analyses of aggregate data make sense. After introducing the notions of structural and empirical aggregation invariance of MPT models, we show that any MPT model that holds at the level of single individuals must also hold at the aggregate level when it is both structurally and empirically aggregation invariant. Moreover, group-level parameters of aggregation-invariant MPT models are equivalent to the expected values (i.e., means) of the corresponding individual parameters. To investigate the robustness of MPT results for aggregate data when one or both invariance conditions are violated, we additionally performed a series of simulation studies, systematically manipulating (1) the sample sizes in different trees of the model, (2) model parameterization, (3) means and variances of crucial model parameters, and (4) their correlations with other parameters of the respective MPT model. Overall, our results show that MPT parameter estimates based on aggregate data are trustworthy under rather general conditions, provided that a few preconditions are met.

Defeat, entrapment and suicidal ideation in a Turkish community sample of young adults: an examination of the Integrated Motivational-Volitional (IMV) model of suicidal behaviour.

The present study examines the relationships between defeat, entrapment, suicidal ideation, thwarted belongingness, and perceived burdensomeness through the Integrated Motivational-Volitional (IMV) Model of Suicidal Behaviour in a sample of Turkish young adults. The sample consisted of 451 individuals (72.5% females, Mage= 25.20). The correlation analyses revealed significant relationships between defeat, entrapment, suicidal ideation, thwarted belongingness, perceived burdensomeness in the expected directions. Mediation and moderation analyses partly confirmed the assumptions of the motivational phase of the IMV model; entrapment played a mediating role between defeat and suicidal ideation, and thwarted belongingness (but not perceived burdensomeness) had a moderating role in the pathway between entrapment and suicidal ideation. These findings add a new dimension to the understanding of suicide risk and potential protective factors through the IMV model, which was tested for the first time in the Turkish population. It is anticipated that this study will contribute to suicide prevention intervention strategies, especially for young adults, at-risk group for suicide in Turkey.

Rethinking Lifetime Abuse in Old Age.

Research on abuse is often piecemeal and focused on specific forms, populations, and social situations. As a result, these studies tend to ignore the web of abuse resulting from the interactive effects of various forms of abuse over the life course, together referred to as "lifetime abuse." An examination of abuse from the point of view of older adults is likely to provide a lifetime perspective based on linking child abuse to elder abuse over the life course and an assessment of its consequences. Despite the growing awareness and resulting body of knowledge regarding this phenomenon, proposed theoretical formulations about it remain limited. The present article aimed to conceptualize the multifaceted phenomenon of lifetime abuse and to gain a deeper perspective of the topic by understanding its dimensions, meanings, and experiences in older age, based on a critical review of empirical studies. A model is suggested, which can be useful for future exploration and understanding of lifetime abuse. Based on a critical review of previous studies, we suggest a theoretical framework of lifetime abuse from the perspective of older adults, identifying three key domains: (a) perception of time and life review in old age; (b) cumulative and interactive abusive life events; and (c) resilience versus vulnerability over the life course. We propose a heuristic model to explore and understand lifetime abuse. We believe our proposed model is open to reflection and elaboration and is intended to raise questions that could enhance the need for our understanding of lifetime abuse in old age.

Psychological Assessment for Gender-Affirming Care in Transgender and Nonbinary Youth: The Companionship Model.

ABSTRACT: This article introduces The Companionship Model as a template for conducting ethical and affirming assessment sessions for transgender and nonbinary youths (TNBY) and their guardians. The Companionship Model emphasizes proactive clinical companionship as TNBY and their guardians navigate access to gender-affirming care. The model offers specific action steps for establishing a therapeutic alliance and centers TNBY and their guardians' access to resources based on the informed consent/assent process. In this article, we provide sample session questions and illustrative clinical examples.

Correlative comparison of visual working memory paradigms and associated models.

When studying the working memory (WM), the 'slot model' and the 'resource model' are two main theories used to describe how information retention occurs. The slot model shows that WM capacity consists of a certain number of predefined slots available for information storage. This theory explains that there is a binary condition during information recall in which information is either wholly maintained within a slot or forgotten. The resource model has a resolution-based approach, suggesting a continuous resource able to be distributed among a number of items in WM capacity. Recently hybrid models have been introduced, suggesting that WM may not strictly conform to only one model. Accordingly, to understand the relationship between two of the most widely used paradigms in WM evaluation, we implemented a correlational assessment in two different psychophysics tasks, an analog recall paradigm with sequential bar presentation and a delayed match-to-sample (DMS) task with checkerboard stimuli. Our study revealed significant correlations between WM performance in the DMS task and recall error, precision, and sources of errors in the sequential paradigm. Overall, the findings emphasize the importance of considering both tasks in understanding WM processes, as they shed light on the debate between the slot and resource models by revealing overlapping elements in both theories and the tasks used to evaluate WM capacity.

A psychological mechanism for the development of anxiety.

Although numerous behavioral constructs have been proposed to account for anxiety disorders, how these disorders develop within an individual has been difficult to predict. In this perspective, I selectively review clinical and experimental evidence suggesting that avoidance (i.e., safety) behavior increases beliefs of threat or fear. The experimental evidence has been replicated numerous times, with different parameters, and shows that when human participants emit avoidance responses in the presence of a neutral stimulus, they later show heightened expectations of threat in the presence of the neutral stimulus. I interpret these findings as resulting from prediction errors as anticipated by the Rescorla-Wagner model, although other animal learning theories can also predict the phenomenon. I discuss some implications and offer a few novel predictions. The analysis presented here sheds light on a phenomenon of theoretical and clinical relevance which is accommodated by basic associative learning theory. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Individuals with anxiety and depression use atypical decision strategies in an uncertain world.

Previous studies on reinforcement learning have identified three prominent phenomena: (1) individuals with anxiety or depression exhibit a reduced learning rate compared to healthy subjects; (2) learning rates may increase or decrease in environments with rapidly changing (i.e. volatile) or stable feedback conditions, a phenomenon termed learning rate adaptation; and (3) reduced learning rate adaptation is associated with several psychiatric disorders. In other words, multiple learning rate parameters are needed to account for behavioral differences across participant populations and volatility contexts in this flexible learning rate (FLR) model. Here, we propose an alternative explanation, suggesting that behavioral variation across participant populations and volatile contexts arises from the use of mixed decision strategies. To test this hypothesis, we constructed a mixture-of-strategies (MOS) model and used it to analyze the behaviors of 54 healthy controls and 32 patients with anxiety and depression in volatile reversal learning tasks. Compared to the FLR model, the MOS model can reproduce the three classic phenomena by using a single set of strategy preference parameters without introducing any learning rate differences. In addition, the MOS model can successfully account for several novel behavioral patterns that cannot be explained by the FLR model. Preferences for different strategies also predict individual variations in symptom severity. These findings underscore the importance of considering mixed strategy use in human learning and decision-making and suggest atypical strategy preference as a potential mechanism for learning deficits in psychiatric disorders.

Risk and resilience models in child development.

This chapter provides the most comprehensive review of risk and resilience models for child development thus far, synthesizing these interdisciplinary frameworks for ease of use in research and practice. This review specifically focuses on process models with broader conceptualizations of risk and resilience that have effects across multiple developmental domains. Risk and resilience models alike agree that alleviating risk factors is beneficial for children's development, including risks ranging from proximal issues with households (e.g., instability) and caregivers (e.g., insecure attachment, abuse) to relatively distal influences like structural racism and socioeconomic status. Resilience models further add that children who experience risks are not inherently doomed to poorer outcomes, but can draw upon positive factors in development to combat negative effects from risk, which cannot always be avoided. Major positive factors include loving relationships, educational resources, and cultural assets. Risk and resilience are highly multidisciplinary fields that have contributed much to our understanding of human development, with ample room for continued growth. Understanding of risk and resilience processes, especially during sensitive developmental periods like early childhood, provides valuable insight for prevention and intervention research and practices. Risk and resilience models share an interest in deciphering the developmental processes that hinder and help children across domains so that kids can live their best lives, resulting in a better off society for all.

Moderate confirmation bias enhances decision-making in groups of reinforcement-learning agents.

Humans tend to give more weight to information confirming their beliefs than to information that disconfirms them. Nevertheless, this apparent irrationality has been shown to improve individual decision-making under uncertainty. However, little is known about this bias' impact on decision-making in a social context. Here, we investigate the conditions under which confirmation bias is beneficial or detrimental to decision-making under social influence. To do so, we develop a Collective Asymmetric Reinforcement Learning (CARL) model in which artificial agents observe others' actions and rewards, and update this information asymmetrically. We use agent-based simulations to study how confirmation bias affects collective performance on a two-armed bandit task, and how resource scarcity, group size and bias strength modulate this effect. We find that a confirmation bias benefits group learning across a wide range of resource-scarcity conditions. Moreover, we discover that, past a critical bias strength, resource abundance favors the emergence of two different performance regimes, one of which is suboptimal. In addition, we find that this regime bifurcation comes with polarization in small groups of agents. Overall, our results suggest the existence of an optimal, moderate level of confirmation bias for decision-making in a social context.