The paradoxical association between tension-reduction alcohol outcome expectancies and tension following alcohol consumption.

Background: Behavioral models suggest that strong tension-reduction alcohol-outcome expectancies (TREs) among drinkers should be associated with greater tension reduction after drinking. Yet, the few studies investigating this have found either no relationship or the opposite relationship.Objectives: We sought to explore this relationship by building upon the limitations of past studies and employing a placebo-controlled, within-subject experimental design.Methods: Sixty social drinkers (26 M, 34 F) visited the lab on two occasions spaced one week apart. Each participant was randomly assigned to receive alcoholic drinks targeting a BAC of 0.05% on one testing day and placebo drinks on the other, with the order counter-balanced. On both testing days, participants completed measures of state anxiety and fear both before drinking and following a drinking/absorption period. While completing the self-report measures, participants were anticipating an impending, mildly stressful heartbeat perception task.Results: Multilevel modeling revealed that the more strongly individuals believed that alcohol reduces tension, the less the pharmacologic properties of alcohol did so (p = .02 for the state anxiety outcome measure; p = .001 for the fear outcome measure). This was the case even with anxiety sensitivity - a known predictor of stress-response dampening - controlled for.Conclusions: These results provide further evidence for the paradoxical association of TREs and the dampening of anxiety. Additionally, the findings are consistent with the basis of expectancy challenges that aim to reframe inaccurate TREs among drinkers.

Personalized Cognitive Health in Psychiatry: Current State and the Promise of Computational Methods.

BACKGROUND: Decades of research have firmly established that cognitive health and cognitive treatment services are a key need for people living with psychosis. However, many current clinical programs do not address this need, despite the essential role that an individual's cognitive and social cognitive capacities play in determining their real-world functioning. Preliminary practice-based research in the Early Psychosis Intervention Network early psychosis intervention network shows that it is possible to develop and implement tools that delineate an individuals' cognitive health profile and that help engage the client and the clinician in shared decision-making and treatment planning that includes cognitive treatments. These findings signify a promising shift toward personalized cognitive health. STUDY DESIGN: Extending upon this early progress, we review the concept of interindividual variability in cognitive domains/processes in psychosis as the basis for offering personalized treatment plans. We present evidence from studies that have used traditional neuropsychological measures as well as findings from emerging computational studies that leverage trial-by-trial behavior data to illuminate the different latent strategies that individuals employ. STUDY RESULT: We posit that these computational techniques, when combined with traditional cognitive assessments, can enrich our understanding of individual differences in treatment needs, which in turn can guide evermore personalized interventions. CONCLUSION: As we find clinically relevant ways to decompose maladaptive behaviors into separate latent cognitive elements captured by model parameters, the ultimate goal is to develop and implement approaches that empower clients and their clinical providers to leverage individual's existing learning capacities to improve their cognitive health and well-being.

Dopamine and norepinephrine differentially mediate the exploration-exploitation tradeoff.

The catecholamines dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision making processes. Although the two neuromodulators share a synthesis pathway and are co-activated under states of arousal, they engage in distinct circuits and roles in modulating neural activity across the brain. However, in the computational neuroscience literature, they have been assigned similar roles in modulating the latent cognitive processes of decision making, in particular the exploration-exploitation tradeoff. Revealing how each neuromodulator contributes to this explore-exploit process will be important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration and exploitation, a direct comparison using the same dynamic decision making task is needed. Here, we ran mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA receptor antagonist (flupenthixol), a nonselective DA receptor agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol), and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine receptor activity on the level of exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. Beta-noradrenergic receptor activity also modulated exploration, but the modulatory effect was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via outcome sensitivity. Together, these findings suggested that the mechanisms that govern the transition between exploration and exploitation are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

Prolonged Physiological Stress Is Associated With a Lower Rate of Exploratory Learning That Is Compounded by Depression.

BACKGROUND: Stress is a major risk factor for depression, and both are associated with important changes in decision-making patterns. However, decades of research have only weakly connected physiological measurements of stress to the subjective experience of depression. Here, we examined the relationship between prolonged physiological stress, mood, and explore-exploit decision making in a population navigating a dynamic environment under stress: health care workers during the COVID-19 pandemic. METHODS: We measured hair cortisol levels in health care workers who completed symptom surveys and performed an explore-exploit restless-bandit decision-making task; 32 participants were included in the final analysis. Hidden Markov and reinforcement learning models assessed task behavior. RESULTS: Participants with higher hair cortisol exhibited less exploration (r = -0.36, p = .046). Higher cortisol levels predicted less learning during exploration (ß = -0.42, false discovery rate [FDR]-corrected p [pFDR] = .022). Importantly, mood did not independently correlate with cortisol concentration, but rather explained additional variance (ß = 0.46, pFDR = .022) and strengthened the relationship between higher cortisol and lower levels of exploratory learning (ß = -0.47, pFDR = .022) in a joint model. These results were corroborated by a reinforcement learning model, which revealed less learning with higher hair cortisol and low mood (ß = -0.67, pFDR = .002). CONCLUSIONS: These results imply that prolonged physiological stress may limit learning from new information and lead to cognitive rigidity, potentially contributing to burnout. Decision-making measures link subjective mood states to measured physiological stress, suggesting that they should be incorporated into future biomarker studies of mood and stress conditions.

Sequential delay and probability discounting tasks in mice reveal anchoring effects partially attributable to decision noise.

Delay discounting and probability discounting decision making tasks in rodent models have high translational potential. However, it is unclear whether the discounted value of the large reward option is the main contributor to variability in animals' choices in either task, which may limit translation to humans. Male and female mice underwent sessions of delay and probability discounting in sequence to assess how choice behavior adapts over experience with each task. To control for "anchoring" (persistent choices based on the initial delay or probability), mice experienced "Worsening" schedules where the large reward was offered under initially favorable conditions that became less favorable during testing, followed by "Improving" schedules where the large reward was offered under initially unfavorable conditions that improved over a session. During delay discounting, both male and female mice showed elimination of anchoring effects over training. In probability discounting, both sexes of mice continued to show some anchoring even after months of training. One possibility is that "noisy", exploratory choices could contribute to these persistent anchoring effects, rather than constant fluctuations in value discounting. We fit choice behavior in individual animals using models that included both a value-based discounting parameter and a decision noise parameter that captured variability in choices deviating from value maximization. Changes in anchoring behavior over time were tracked by changes in both the value and decision noise parameters in delay discounting, but by the decision noise parameter in probability discounting. Exploratory decision making was also reflected in choice response times that tracked the degree of conflict caused by both uncertainty and temporal cost, but was not linked with differences in locomotor activity reflecting chamber exploration. Thus, variable discounting behavior in mice can result from changes in exploration of the decision options rather than changes in reward valuation.

Sex differences in learning from exploration.

Sex-based modulation of cognitive processes could set the stage for individual differences in vulnerability to neuropsychiatric disorders. While value-based decision making processes in particular have been proposed to be influenced by sex differences, the overall correct performance in decision making tasks often show variable or minimal differences across sexes. Computational tools allow us to uncover latent variables that define different decision making approaches, even in animals with similar correct performance. Here, we quantify sex differences in mice in the latent variables underlying behavior in a classic value-based decision making task: a restless two-armed bandit. While male and female mice had similar accuracy, they achieved this performance via different patterns of exploration. Male mice tended to make more exploratory choices overall, largely because they appeared to get 'stuck' in exploration once they had started. Female mice tended to explore less but learned more quickly during exploration. Together, these results suggest that sex exerts stronger influences on decision making during periods of learning and exploration than during stable choices. Exploration during decision making is altered in people diagnosed with addictions, depression, and neurodevelopmental disabilities, pinpointing the neural mechanisms of exploration as a highly translational avenue for conferring sex-modulated vulnerability to neuropsychiatric diagnoses.

When faced with a decision to make, humans and other animals reflect on past experiences of similar situations to choose the best option. However, in an uncertain situation, this decision process requires balancing two competing priorities: exploiting options that are expected to be rewarding (exploitation), and exploring alternatives that could be more valuable (exploration). Decision making and exploration are disrupted in many mental disorders, some of which can differ in either presentation or risk of development across women and men. This raises the question of whether sex differences in exploration and exploitation could contribute to the vulnerability to these conditions. To shed light on this question, Chen et al. studied exploration in male and female mice as they played a video game. The mice had the option to touch one of two locations on a screen for a chance to win a small reward. The likelihood of success was different between the two options, and so the mice were incentivized to determine which was the more rewarding button. While the mice were similarly successful in finding rewards regardless of sex, on average male mice were more likely to keep exploring between the options while female mice more quickly gained confidence in an option. These differences were stronger during uncertain periods of learning and exploration than when making choices in a well-known situation, indicating that periods of uncertainty are when the influence of sex on cognition are most visible. However, not every female or male mouse was the same ­ there was as much variability within a sex as was seen between sexes. These results indicate that sex mechanisms, along with many other influences cause individual differences in the cognitive processes important for decision making. The approach used by Chen et al. could help to study individual differences in cognition in other species, and shed light on how individual differences in decision-making processes could contribute to risk and resilience to mental disorders.

Divergent Strategies for Learning in Males and Females.

A frequent assumption in value-based decision-making tasks is that agents make decisions based on the feature dimension that reward probabilities vary on. However, in complex, multidimensional environments, stimuli can vary on multiple dimensions at once, meaning that the feature deserving the most credit for outcomes is not always obvious. As a result, individuals may vary in the strategies used to sample stimuli across dimensions, and these strategies may have an unrecognized influence on decision-making. Sex is a proxy for multiple genetic and endocrine influences on behavior, including how environments are sampled. In this study, we examined the strategies adopted by female and male mice as they learned the value of stimuli that varied in both image and location in a visually cued two-armed bandit, allowing two possible dimensions to learn about. Female mice acquired the correct image-value associations more quickly than male mice, preferring a fundamentally different strategy. Female mice were more likely to constrain their decision-space early in learning by preferentially sampling one location over which images varied. Conversely, male mice were more likely to be inconsistent, changing their choice frequently and responding to the immediate experience of stochastic rewards. Individual strategies were related to sex-biased changes in neuronal activation in early learning. Together, we find that in mice, sex is associated with divergent strategies for sampling and learning about the world, revealing substantial unrecognized variability in the approaches implemented during value-based decision making.