Reinforcement learning in and out of context: The effects of attentional focus.

In reinforcement learning (RL) tasks, decision makers learn the values of actions in a context-dependent fashion. Although context dependence has many advantages, it can lead to suboptimal preferences when choice options are extrapolated beyond their original encoding contexts. Here, we tested whether we could manipulate context dependence in RL by introducing a secondary task designed to bias attention toward either absolute or relative outcomes. Participants completed a learning phase that involved choices between two (Experiment 1; n = 111) or three (Experiment 2; n = 90) options per trial with complete feedback. Choice options were grouped in stable contexts so that only a small set of the possible combinations were encountered. One group of participants rated how they felt about particular options (Feelings condition), and another group reported how much they expected to win from particular options (Outcomes condition) at occasional points throughout the learning phase. A third group (Control condition) made no ratings. In the subsequent transfer test, participants chose between all possible pairs of options without feedback. The experimental manipulation had no effect on learning phase performance but a significant effect on transfer, with the Feelings and Control conditions exhibiting greater context dependence than the Outcomes condition. Further, rated feelings reflected relative valuation whereas expected outcomes were more sensitive to absolute option values. Hierarchical Bayesian modeling was used to summarize the findings from both experiments. Our results suggest that attending to affective reactions versus expected outcomes moderates the effects of encoding context on subsequent choices. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Testing models of context-dependent outcome encoding in reinforcement learning.

Previous studies of reinforcement learning (RL) have established that choice outcomes are encoded in a context-dependent fashion. Several computational models have been proposed to explain context-dependent encoding, including reference point centering and range adaptation models. The former assumes that outcomes are centered around a running estimate of the average reward in each choice context, while the latter assumes that outcomes are compared to the minimum reward and then scaled by an estimate of the range of outcomes in each choice context. However, there are other computational mechanisms that can explain context dependence in RL. In the present study, a frequency encoding model is introduced that assumes outcomes are evaluated based on their proportional rank within a sample of recently experienced outcomes from the local context. A range-frequency model is also considered that combines the range adaptation and frequency encoding mechanisms. We conducted two fully incentivized behavioral experiments using choice tasks for which the candidate models make divergent predictions. The results were most consistent with models that incorporate frequency or rank-based encoding. The findings from these experiments deepen our understanding of the underlying computational processes mediating context-dependent outcome encoding in human RL.

Effects of blocked versus interleaved training on relative value learning.

In reinforcement learning tasks, people learn the values of options relative to other options in the local context. Prior research suggests that relative value learning is enhanced when choice contexts are temporally clustered in a blocked sequence compared to a randomly interleaved sequence. The present study was aimed at further investigating the effects of blocked versus interleaved training using a choice task that distinguishes among different contextual encoding models. Our results showed that the presentation format in which contexts are experienced can lead to qualitatively distinct forms of relative value learning. This conclusion was supported by a combination of model-free and model-based analyses. In the blocked condition, choice behavior was most consistent with a reference point model in which outcomes are encoded relative to a dynamic estimate of the contextual average reward. In contrast, the interleaved condition was best described by a range-frequency encoding model. We propose that blocked training makes it easier to track contextual outcome statistics, such as the average reward, which may then be used to relativize the values of experienced outcomes. When contexts are interleaved, range-frequency encoding may serve as a more efficient means of storing option values in memory for later retrieval.

The effect of auditory valence on subsequent visual semantic processing.

Emotion influences many cognitive processes and plays an important role in our daily life. Previous studies focused on the effects of arousal on subsequent cognitive processing, but the effect of valence on subsequent semantic processing is still not clear. The present study examined the effect of auditory valence on subsequent visual semantic processing when controlling for arousal. We used instrumental music clips varying in valence while matching in arousal to induce valence states and asked participants to make natural or man-made judgements on subsequent neutral objects. We found that positive and negative valences similarly impaired subsequent semantic processing compared with neutral valence. The linear ballistic accumulator model analyses showed that the valence effects can be attributed to drift rate differences, suggesting that the effects are likely related to attentional selection. Our findings are consistent with a motivated attention model, indicating comparable attentional capture by both positive and negative valences in modulating subsequent cognitive processes.

Bird communities across different levels of human settlement: A comparative analysis from two northern Amazonian ecoregions.

Urban ecosystems are increasingly dominating landscapes globally, so it is critical to understand the effects of human settlements on biodiversity. Bird communities are effective indicators because they are impacted by the size and expansion of human settlements, exemplified by changes in their habitat use, breeding and foraging behaviours, as well as patterns of richness and abundance. Existing studies on bird community responses to human settlements have mainly focused on single ecoregions and large cities, leaving a gap in comparative research on how differently sized human settlements affect bird communities across various ecoregions. To address this gap, we examine species richness, bird abundances and community composition in human settlements, which exhibit variable sizes, populations, landscape configurations, and overall intensity of settlement in two tropical ecoregions in Guyana, Amazonia: forest and savannah. In each ecoregion we explored how different groupings of urban tolerance in birds responded to human settlements of differing population size and building densities. Overall, we found significant differences in bird communities across the varying levels of human settlement intensity in both ecoregions, with greater differences in bird community composition in the forest ecoregion than the savannah region. In both ecoregions, species richness and abundance were highest at the medium level of settlement of human settlement. Our findings suggest that bird tolerance to human settlements varies based on ecoregion and site-level factors. In the savannah, built features may be benefitting birds from all urban tolerance levels, but they have a negative impact on less urban-tolerant species in the forest ecoregion. Our comparative analysis reveals for the first time that the impact of human settlements on avian communities in northern Amazonia varies among ecoregions, indicating that species evolved to live in a savannah may be more tolerant to human settlements than those more evolved to a forest system.

Context effects on choice under cognitive load.

The context-dependent nature of choice is well illustrated by decoy effects, in which adding an alternative to a choice set can change the preference relations among the other alternatives. The current within-subjects study tested whether manipulating cognitive load affects the magnitude of attraction and compromise decoy effects. Participants (n = 96) made simulated online grocery shopping choices from three options described by price and quality for each grocery item they encountered. On half the 96 trials, they had to memorize a telephone number prior to encountering the choice set, after which they recalled the number. The choice task was rated significantly more difficult under load, providing some face validity for the load manipulation. Across decoy types, context effects were large and unaffected by the load manipulation. Bayesian analysis provided substantial evidence in favor of this null effect, with the study powered at better than .95 to detect a moderate effect. Across individuals, the magnitude of decoy effects was positively correlated with perception of the greater difficulty of the task under load, with this relationship fully mediated by increases in response times. These results are consistent with the idea that compromise and attraction decoy effects can operate relatively automatically and require minimal effortful processing.

Bird diversity and psychological wellbeing: A comparison of green and coastal blue space in a neotropical city.

Accelerating rates of urbanisation are contributing to biodiversity declines worldwide. However, urban green (e.g. parks) and blue spaces (e.g. coast) provide important habitat for species. Emerging evidence also shows that green and blue spaces can benefit human psychological wellbeing, although few studies originate from the Global South and it is unclear whether more biodiverse spaces offer greater wellbeing gains. We examine how bird diversity (abundance, species richness, Shannon diversity, and community composition) in green and coastal blue space in Georgetown, Guyana, is associated with people's wellbeing (positive and negative affect, anxiety) in situ, using point counts and questionnaires. Bird community composition differed between green and coastal sites, and diversity was significantly higher in green sites. Positive affect and anxiety did not differ between green and coastal sites, but negative affect was higher in coastal sites. Mixed-effect models showed no associations between biodiversity and wellbeing, implying other features are contributing to people's positive wellbeing. Despite no association between biodiversity and wellbeing, both green and coastal blue sites are important for wellbeing and supporting different bird communities. City planning authorities and public health professionals should ensure these social and environmental needs are met in developing cities in the Global South.

Autonomic responses to choice outcomes: Links to task performance and reinforcement-learning parameters.

Previous work has shown that autonomic responses to choice feedback can predict subsequent decision-making. In this study, we tested whether skin conductance responses (SCRs) and heart rate (HR) decelerations following the presentation of choice outcomes predict Iowa Gambling Task performance in nonclinical participants (n = 64). We also examined how these signals related to parameters of a reinforcement-learning (RL) model. Feedback SCRs and HR decelerations were greater following outcomes that included losses and choices from the bad decks defined by their negative expected value. In addition, SCRs predicted task performance. A hierarchical Bayesian RL model indicated that greater feedback SCR for the bad decks compared to good decks was associated with stronger loss aversion and a lower learning rate, both of which predicted higher performance. These results suggest that feedback-related SCRs are linked to individual differences in outcome evaluation and learning processes that guide reinforcement-learning.

Correction to: Context effects on reproduced magnitudes from short-term and long-term memory.

The citation of Hellström (1985) in the body and the reference section of this article is incorrectly printed as Helström.

Context effects on reproduced magnitudes from short-term and long-term memory.

Extant research has demonstrated strong contextual dependencies in reproducing magnitudes of perceptual stimuli from short-term memory. Two experiments examined how context as defined by (a) the mean of the distribution, (b) stimulus ranks, (c) values of anchor stimuli used in the reproduction task, and (d) values from the most recent trial operate on estimates of square size. Experiment 1 demonstrated distributional contrast effects on ratings of squares and distributional assimilation effects on reproduction of squares from short-term memory for the same participants. The fit of a modified version of the category adjustment model demonstrated reliable effects of the running mean, start anchors, and previous stimulus on reproduction bias. In Experiment 2, participants first learned to associate labels with squares, then reproduced square sizes based on the label cues, a long-term memory task, followed by a reproduction from short-term memory task as in Experiment 1. Results for the short-term memory task were largely consistent with Experiment 1. Results for the long-term memory task showed a very different pattern of effects, with larger reproduced sizes when squares were drawn from positively skewed rather than negatively skewed distributions. This contrast effect was explained by a modified range-frequency model as the result of rank encoding of square values along with displacement away from the running mean and shifts towards the prior response and start anchors. The combined results identify multiple sources of context effects in estimation that depend critically on memory retrieval factors and show how they can be incorporated into existing models.

Context effects in reproduction and perception of song tempo.

Studies of tempo perception suggest that exposure to a distribution of predominantly faster or slower versions of a song can shift one's memory for the original tempo toward the contextual tempos. Three experiments were conducted to examine whether similar assimilation effects would occur when participants are asked to reproduce the tempo of a song from memory. In Experiments 1 and 2, participants listened to a tempo-altered version of a pop song prior to tapping out the original tempo from memory on each trial. Reproduced tempos assimilated toward the immediately preceding tempo, but there was no evidence of global assimilation toward the mean of the distribution of tempos. However, Experiment 2 demonstrated a partial dissociation between perception and production, with the same participants showing large assimilation effects derived from comparative judgments but not from tempo reproduction. In Experiment 3, participants listened to and then tapped out the beat of a tempo-altered version before reproducing the original from memory on each trial, which resulted in a global assimilation effect in reproduction. The results of these experiments highlight that contextual bias in memory for tempo depends on the match between the context and the task, with differential effects for perceptual and motor contexts. (PsycINFO Database Record (c) 2020 APA, all rights reserved).